Ottmar’s Notes - Generator

Create fast insert scripts

2021-01-31T00:00:00+00:00

A few weeks ago I released a new version of my PLEX project</a>. I improved the ability to export table data as insert scripts. Previously, PLEX was only able to export CSV data. The reason was that I originally developed PLEX to speed up the implementation of version control in Oracle DB projects, and the ability to export CSV data was intended to track changes in master data tables.</p>

It turned out that a colleague of mine wanted PLEX to export insert scripts for all data in all tables for the initial deployment of a new application. I implemented an initial version, but was not happy with the runtime of the insert scripts. I now hear some people arguing that there are better tools than insert scripts for importing larger amounts of data - but sometimes this is the only accepted way because only SQL*Plus is available to run the deployment scripts….</p>

Then I stumbled across an older post by Connor McDonald on creating fast insert scripts</a>. I have integrated his performance hacks into PLEX.</p>

You can use them as follows (see last parameter):</p>

declare</span></span>
  l_zip_file blob;</span></span>
begin</span></span>
  l_zip_file :</span>=</span> plex.to_zip(plex.backapp(</span></span>
    p_app_id                   </span>=></span> 100</span>,  </span>-- parameter only available when APEX is installed</span></span>
    p_include_ords_modules     </span>=></span> true, </span>-- parameter only available when ORDS is installed</span></span>
    p_include_object_ddl       </span>=></span> true,</span></span>
    p_include_data             </span>=></span> true,</span></span>
    p_data_as_of_minutes_ago   </span>=></span> 0</span>,     </span>-- Read consistent data with the resulting timestamp (SCN). Defaults to 0.</span></span>
    p_data_max_rows            </span>=></span> 10000</span>, </span>-- Maximum number of rows per table. Defaults to 1000.</span></span>
    p_data_table_name_like     </span>=></span> 'CAT\_%,USERS,ROLES,RIGHTS'</span>,</span></span>
    p_data_table_name_not_like </span>=></span> '%\_HIST,LOGGER%'</span>,</span></span>
    p_data_format              </span>=></span> 'insert:10'</span>));</span></span>
</span>
  -- Do something with the zip file.</span></span>
  -- Your code here...</span></span>
end</span>;</span></span>
/</span></span></code></pre>
The p_data_format</code> parameter is new and can contain a comma-separated list of formats - currently the CSV and INSERT formats are supported. An example: csv,insert</code> exports for each table a CSV file and a SQL file with insert statements. For insert you can also specify the number of rows per insert all</code> statement (default value is 20) - example: csv,insert:10</code> or insert:5</code>.</p>
As you can see from the p_data_table_name_not_like => '%\_HIST,LOGGER%'</code> parameter, PLEX also accepts lists with like filters. In our example %\_HIST,LOGGER%</code> is translated to:</p>
where</span> ...</span></span>
  and</span> (    table_name </span>not like</span> '%\_HIST'</span> escape </span>'\'</span></span>
       and</span> table_name </span>not like</span> 'LOGGER%'</span> escape </span>'\'</span></span>
       and</span> ...</span></span>
      )</span></span></code></pre>
For the parameter p_data_table_name_like => 'CAT\_%,USERS,ROLES,RIGHTS'</code> the tables are filtered as follows:</p>
where</span> ...</span></span>
  and</span> (   table_name </span>like</span> 'CAT\_%'</span> escape </span>'\'</span></span>
       or</span> table_name </span>like</span> 'USERS'</span>  escape </span>'\'</span></span>
       or</span> ...</span></span>
      )</span></span></code></pre>
There is a good reason why the default p_data_max_rows</code> parameter is set to 1000</code> - if you don’t specify table name filters, all tables will be exported…</p>
The resulting insert scripts look like this:</p>
-- Script generated by PLEX version 2.4.2 - more infos here: https://github.com/ogobrecht/plex</span></span>
-- Performance Hacks by Connor McDonald: https://connor-mcdonald.com/2019/05/17/hacking-together-faster-inserts/</span></span>
-- For strange line end replacements a big thank to SQL*Plus: https://support.oracle.com/epmos/faces/DocumentDisplay?id=2377701.1 (SQL Failed With ORA-1756 In Sqlplus But Works In SQL Developer)</span></span>
prompt </span>-</span> insert</span> 847</span> rows into</span> USERS (exported </span>2021</span>-</span>01</span>-</span>31 14</span>:</span>24</span>:</span>30</span>.</span>703234000</span> +</span>00</span>:</span>00</span>)</span></span>
set</span> define </span>off</span> feedback </span>off</span> sqlblanklines </span>on</span></span>
alter session set</span> cursor_sharing          </span>=</span> force</span>;</span></span>
alter session set</span> nls_numeric_characters  </span>=</span> '.,'</span>;</span></span>
alter session set</span> nls_date_format         </span>=</span> 'yyyy-mm-dd hh24:mi:ss'</span>;</span></span>
alter session set</span> nls_timestamp_format    </span>=</span> 'yyyy-mm-dd hh24:mi:ssxff'</span>;</span></span>
alter session set</span> nls_timestamp_tz_format </span>=</span> 'yyyy-mm-dd hh24:mi:ssxff tzr'</span>;</span></span>
begin</span></span>
insert</span> all</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>1</span>,</span>'Ylcmsajbil'</span>,</span>'Fojkjryntnixzfh'</span>,</span>'qvspjgvwmtbi@ghovilkddx.mly'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>2</span>,</span>'Axify'</span>,</span>'Cofjlkwzxytdih'</span>,</span>'ajgttnqlds@minokpyfo.gu'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>3</span>,</span>'Ckyqqvuqrkuktb'</span>,</span>'Igacqwp'</span>,</span>'qpygabuhbrs@nsjxpgjlle.ze'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>4</span>,</span>'Noythhl'</span>,</span>'Gausfu'</span>,</span>'ngmgsbr@duyxqzn.hmyo'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>5</span>,</span>'Kpewbrecnfzsi'</span>,</span>'Nwbsnjh'</span>,</span>'xwlhcfaxko@uhqsibdojjp.hsm'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>6</span>,</span>'Qlzyizkl'</span>,</span>'Gwnaojlvyud'</span>,</span>'kzndqj@nsosenf.fm'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>7</span>,</span>'Xpepttcemrd'</span>,</span>'Ktaqqdnqyfvc'</span>,</span>'uhbnzezvz@buiptt.lkrm'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>8</span>,</span>'Ybyinr'</span>,</span>'Vngairocujhy'</span>,</span>'igvfzoegbh@hsepkqiwbst.evs'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>9</span>,</span>'Ctjixltj'</span>,</span>'Yvsiei'</span>,</span>'ozpspssyw@vooiyfuf.xeh'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>10</span>,</span>'Lttzwsavnozxu'</span>,</span>'Kcyjalvzrl'</span>,</span>'yvwowaqrpku@dyapdumb.fvi'</span>)</span></span>
select</span> *</span> from</span> dual;</span></span>
</span>
/* S N I P */</span></span>
</span>
insert</span> all</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>841</span>,</span>'Gpkabykoveq'</span>,</span>'Gljhlrijqop'</span>,</span>'imnhrheyr@ypccyiu.ah'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>842</span>,</span>'Mwrfazphbvmekpw'</span>,</span>'Kxirzfth'</span>,</span>'fxoatt@frlbwbn.tf'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>843</span>,</span>'Hzxomvkliaxl'</span>,</span>'Mstdrrmgfmsy'</span>,</span>'gpeidglzwfa@hwyumsansy.fet'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>844</span>,</span>'Vwvwezfsd'</span>,</span>'Xtfouojiymtlu'</span>,</span>'zgsdtowsvt@ywfngnijgts.ozd'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>845</span>,</span>'Eljntjfkxx'</span>,</span>'Sifgii'</span>,</span>'gksggat@ubfmmdopqy.ly'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>846</span>,</span>'Ewvyjeudjb'</span>,</span>'Anihlpdgeg'</span>,</span>'gietuk@ezciwejuedy.nuf'</span>)</span></span>
into</span> USERS(U_ID,U_FIRST_NAME,U_LAST_NAME,U_EMAIL) </span>values</span> (</span>847</span>,</span>'Wscdamsgssmouf'</span>,</span>'Omtaofvlrjs'</span>,</span>'jrwyzftmbmo@gjylnuez.esq'</span>)</span></span>
select</span> *</span> from</span> dual;</span></span>
end</span>;</span></span>
/</span></span>
commit</span>;</span></span>
alter session set</span> cursor_sharing </span>=</span> exact;</span></span>
set</span> define </span>on</span></span></code></pre>
PLEX saves the NLS settings of the current session and makes sure that the insert scripts run with the same settings - this means that you should not get any number or date conversion errors. The performance hacks from Connor are alter session set cursor_sharing = force;</code> and insert all</code> - thanks for pointing that out Connor :-)</p>
PLEX has many parameters to configure the export - the focus of PLEX is to quickly introduce version control to existing projects (see also this article about PLEX in general</a>). If you want to use PLEX only for exporting table data, then you should create a wrapper function for it that suits your personal needs - here is an example:</p>
create or replace function</span> to_insert</span>(</span></span>
  p_table_name </span>varchar2</span>,</span></span>
  p_max_rows   </span>integer default</span> 1000</span>,</span></span>
  p_batch_size </span>integer default</span> 20</span>)</span></span>
  return</span> clob</span></span>
is</span></span>
  l_file_collection plex.tab_export_files;</span></span>
begin</span></span>
  l_file_collection :</span>=</span> plex.backapp(</span></span>
    p_include_data         </span>=></span> true,</span></span>
    p_data_max_rows        </span>=></span> p_max_rows,</span></span>
    p_data_table_name_like </span>=></span> p_table_name,</span></span>
    p_data_format          </span>=></span> 'insert:'</span> ||</span> to_char(p_batch_size),</span></span>
    --overwrite defaults to save work for plex:</span></span>
    p_include_templates    </span>=></span> false,</span></span>
    p_include_runtime_log  </span>=></span> false,</span></span>
    p_include_error_log    </span>=></span> false);</span></span>
  return</span> l_file_collection(</span>1</span>).contents;</span></span>
end</span>;</span></span>
/</span></span></code></pre>
Now you can always do a quick export like this: select to_insert('USERS', 10000) from dual;</code></p>
Happy exporting and scripting :-)</p>
Ottmar</p>



New Version of Oracle Table API Generator Available
2020-12-26T00:00:00+00:00
Short before Christmas André and I released version 0.6.0 of our table API generator</a>. The last release was exactly two years ago - time flies. This blog post highlights some of the changes - the complete changelog is available on GitHub</a>.</p>
Bulk Processing</h2>
When you split your app over multiple schemas for higher data security you might need at some point the possibility to bulk process your data. Imagine the following, common schema setup (also see this nice video by Connor McDonald</a>, 20 minutes):</p>

Data Schema: Holds your tables and provides table APIs for the other schemas, which have only select rights on the tables (or views) and execute rights on the table APIs</li>
API Schema: Holds your business logic and provides business APIs to the app parsing schema</li>
UI Schema: Has no own objects, only connect rights and is used as the parsing schema to run the application</li>
</ul>
This setup follows the principle of the least privilege. When you need to bulk process your data in the business logic (API schema), then you have a problem without bulk processing facilities in your table APIs. You need to implement business logic in the data schema or you grant update and/or delete rights to the API schema. The first solution is against the separation of concerns and the second one lowers your data security and maybe prevents you from not having triggers (more about this later in the audit columns section).</p>
That is why we have now also bulk processing in the core of our generated table APIs. Here is how you can use it in your business logic:</p>
/* EXAMPLE INSERT</span></span>
To keep the memory usage low we break our inserts into</span></span>
a bulk size of 1000 rows. You should adapt the bulk size</span></span>
to your use case. */</span></span>
declare</span></span>
  l_rows_tab     data_schema.your_table_api.t_rows_tab;</span></span>
  l_number_bulks </span>integer</span> :</span>=</span> 100</span>;</span></span>
  l_bulk_size    </span>integer</span> :</span>=</span> 1000</span>;</span></span>
begin</span></span>
  l_rows_tab :</span>=</span> data_schema.your_table_api.t_rows_tab</span>()</span>;</span></span>
  l_rows_tab.extend(l_bulk_size);</span></span>
</span>
  <<</span>number_bulks</span>>></span></span>
  for</span> z </span>in</span> 1</span> .. l_number_bulks </span>loop</span></span>
</span>
    <<</span>bulk_size</span>>></span></span>
    for</span> i </span>in</span> 1</span> .. l_bulk_size </span>loop</span></span>
      l_rows_tab(i).column_1 :</span>=</span> your_logic_here;</span></span>
      l_rows_tab(i).column_2 :</span>=</span> your_logic_here;</span></span>
    end loop</span> bulk_size;</span></span>
</span>
    data_schema.your_table_api.create_rows(l_rows_tab);</span></span>
    commit</span>;</span></span>
</span>
  end loop</span> number_bulks;</span></span>
end</span>;</span></span>
/</span></span></code></pre>/* EXAMPLE UPDATE */</span></span>
declare</span></span>
  l_rows_tab   data_schema.your_table_api.t_rows_tab;</span></span>
  l_ref_cursor data_schema.your_table_api.t_strong_ref_cursor;</span></span>
begin</span></span>
  -- optionally set bulk limit, default is 1000</span></span>
  -- data_schema.your_table_api.set_bulk_limit(500);</span></span>
  open</span> l_ref_cursor </span>for</span></span>
    select</span> *</span> from</span> app_users </span>where</span> your_conditions_here;</span></span>
</span>
  <<</span>outer_bulk</span>>></span></span>
  loop</span></span>
    l_rows_tab :</span>=</span> data_schema.your_table_api.read_rows(l_ref_cursor);</span></span>
</span>
    <<</span>inner_data</span>>></span></span>
    for</span> i </span>in</span> 1</span> .. l_rows_tab.count</span></span>
    loop</span></span>
      --do your business logic here</span></span>
      l_rows_tab(i).column_1 :</span>=</span> your_logic_here;</span></span>
    end loop</span> inner_data;</span></span>
</span>
    data_schema.your_table_api.update_rows(l_rows_tab);</span></span>
    commit</span>;</span></span>
</span>
    exit </span>when</span> data_schema.your_table_api.bulk_is_complete;</span></span>
  end loop</span> outer_bulk;</span></span>
</span>
  close</span> l_ref_cursor;</span></span>
end</span>;</span></span>
/</span></span></code></pre>
The bulk deletion works the same way as the bulk update - the difference is that you do not modify your data before calling the delete_rows</code> method.</p>
For more examples see the docs:</p>

Basic usage</a></li>
Performance tests</a></li>
</ul>
Support for Audit Columns</h2>
Keeping the bulk processing in mind for performance reasons it is a bad idea to have triggers only to manage audit columns in a table. If you have your tables in an own schema and the business logic and the app cannot directly write to the tables then it is easy to prevent triggers. Therefore the table APIs need to support that and hides the audit columns for insert or update operations. This is what our generator now supports.</p>
You can map your specific audit column names to the four often used names created</code>, created_by</code>, updated</code> and updated_by</code> by providing a mapping string to the new parameter p_audit_column_mappings</code> - an example:</p>
-- TABLE DEFINITION --</span></span>
create table</span> app_users</span> (</span></span>
  au_id          </span>integer            generated always as identity</span>,</span></span>
  au_first_name  </span>varchar2</span>(</span>15</span> char</span>)                         ,</span></span>
  au_last_name   </span>varchar2</span>(</span>15</span> char</span>)                         ,</span></span>
  au_email       </span>varchar2</span>(</span>30</span> char</span>)               </span>not null</span>  ,</span></span>
  au_active_yn   </span>varchar2</span>(</span>1</span> char</span>)   </span>default</span> 'Y'</span>  not null</span>  ,</span></span>
  -- This is only for demo purposes. In reality we expect</span></span>
  -- more unified names and types for audit columns.</span></span>
  au_created_on  </span>date                            not null</span>  ,</span></span>
  au_created_by  </span>char</span>(</span>15</span> char</span>)                   </span>not null</span>  ,</span></span>
  au_updated_at  </span>timestamp                       not null</span>  ,</span></span>
  au_updated_by  </span>varchar2</span>(</span>15</span> char</span>)               </span>not null</span>  ,</span></span>
  --</span></span>
  primary key</span> (au_id),</span></span>
  unique</span> (au_email),</span></span>
  check</span> (au_active_yn </span>in</span> (</span>'Y'</span>, </span>'N'</span>))</span></span>
);</span></span>
</span>
begin</span></span>
  om_tapigen.compile_api(</span></span>
    p_table_name            </span>=></span> 'APP_USERS'</span>,</span></span>
    p_audit_column_mappings </span>=></span> 'created=#PREFIX#_CREATED_ON, created_by=#PREFIX#_CREATED_BY, updated=#PREFIX#_UPDATED_AT, updated_by=#PREFIX#_UPDATED_BY'</span></span>
  );</span></span>
end</span>;</span></span>
/</span></span></code></pre>
The #PREFIX#</code> in our example is optional (determined automatically by the generator) and replaced with the found column_prefix au</code>. Doing it this way you can provide always the same audit column mapping to all of your tables. I know some people hate column prefixes - but if you design your data models this way the generator can save you the work to align the mapping for each table.</p>
Support for a Row Version Column</h2>
The same reasons with triggers for the audit columns are valid for the support of a row version column. This can also be handled by our generator:</p>
begin</span></span>
  om_tapigen.compile_api(</span></span>
    p_table_name                 </span>=></span> 'YOUR_TABLE'</span>,</span></span>
    p_row_version_column_mapping </span>=></span> '#PREFIX#_VERSION_ID=tag_global_version_sequence.nextval'</span></span>
  );</span></span>
end</span>;</span></span>
/</span></span></code></pre>
You can also see here the before mentioned #PREFIX#</code> substitution. The difference is that you define here only your column and for the column the SQL expression which should be used for the row version column - in this case, a sequence.</p>
What we cannot do here is to add always 1 to the previous value like it is possible in a trigger - the table API has no idea what the previous value was. So, we are a little bit limited to a SQL expression but the point is, you can let the generated API manage the row version column without having a trigger.</p>
Tenant Support</h2>
This is the third new column mapping definition. For all people who have not an enterprise version with the VPD (virtual private database) functionality you can define a tenant expression for one of your columns:</p>
begin</span></span>
  om_tapigen.compile_api(</span></span>
    p_table_name            </span>=></span> 'YOUR_TABLE'</span>,</span></span>
    p_tenant_column_mapping </span>=></span> '#PREFIX#_TENANT_ID=to_number(sys_context(''my_sec_ctx'',''my_tenant_id''))'</span></span>
  );</span></span>
end</span>;</span></span>
/</span></span></code></pre>
For more information see the parameter docs for p_tenant_column_mapping</a>.</p>
More</h2>

A new optional 1:1 view can be generated (new parameter p_enable_one_to_one_view</code>)</li>
All generated object names can now be customized (new parameters p_dml_view_name</code>, p_dml_view_trigger_name</code> and p_one_to_one_view_name</code>)</li>
Double quoting of table and column names in the generated code can now be configured (new parameter p_double_quote_names</code>, default true)</li>
A new helper function get_default_row</code> will be generated, when parameter p_enable_column_defaults</code> is set to true (to support the row based create_row function)</li>
We have now unit tests driven by utPLSQL (it will be a permanent task to improve the tests with every new feature or bugfix)</li>
</ul>
Breaking Changes</h2>
Where there is light, there is also shadow.</p>
All the new features like bulk processing and audit column support make it hard, if not impossible, to generate a generic change log during the row processing. Also the check, if we have a modification and omit an update if not necessary is hard to implement and not good for the performance with the evolved generator. So we skipped the mentioned functionality.</p>
As far as we know not many people use the generic changelog. If you do, you have to stick with the previous generator version or you have to find an alternative way to do it in the future. A good way seems to be the use of the Flashback Data Archive as it is performant and available in all database editions for free - and you can load existing data into the archive (see Tims article below):</p>

Blog post by Connor McDonald: Triggers can provide auditing information, but there’s a future in flashback</a></li>
Blog post by Tim Hall: Flashback Data Archive (FDA) Enhancements in Oracle Database 12c Release 1 (12.1)</a></li>
Video by Connor McDonald: Flashback Data Archive</a> (14 minutes)</li>
Video APEX Connect 2020 by Connor McDonald: Flashback – so much more</a> (47 minutes)</li>
Video AskTOM Office Hours by Chris Saxon: The Incredible Flashback Data Archive</a> (53 minutes)</li>
</ul>
And the last Breaking change is, that you need at least Oracle DB 12.1 or higher. That should (hopefully) no problem for the most projects.</p>
Conclusion</h2>
A big THANK YOU to all contributors - the current version of the generator would not be possible without your feedback and help.</p>
Be sure to test the new generator before migrating to it. We use it already in some projects and it should be pretty stable. Nevertheless, with so many changed things we cannot give you a guarantee for every edge case - by using it you accept the MIT license. Please check out the project page at GitHub</a> and open an issue</a> if you encounter a problem.</p>
A happy new year and happy coding :-)</p>
Ottmar</p>


PLOC - Convert PL/SQL Code into Markdown Docs
2019-01-10T00:00:00+00:00
</p>
Some months ago I had to write a README file with Markdown syntax for a PL/SQL open source project. I changed function and procedure parameters of the package often during the implementation. Therefore, maintaining the documentation was no fun in the beginning. Maybe you know this from your own (payed) projects ;-)</p>
Normally in the business world I see minimalistic comments in the code and a dedicated documentation independent of the code. As you can imagine, the latter is sometimes - let’s say “out of date”, if it exists at all …</p>
I don’t want to discuss the reasons for this here - you all know several ones. And I also do not have a general solution for code documentation. But I have some ideas what is important for me:</p>

The base of a good PL/SQL code documentation is a good code structure

No single functions and procedures - packages with meaningful names are the way to go</li>
</ul>
</li>
As I have the source code under version control in local files on my computer I need a command line tool for the doc generation - there is no need for a database connection

The CLI should be able to process multiple files (packages)</li>
For my small open source project I have only one package and I want to define a document name different then the package name, so I need a CLI parameter for that</li>
</ul>
</li>
Comments should be written directly after the package/function/procedure signature (in the package spec, optionally it should also work for triggers and types)

I want to use Markdown for the comments - this allows me to write also bigger text parts and code examples</li>
</ul>
</li>
I don’t want to describe all parameters in a JavaDoc manner

PL/SQL is strong typed, everything I need to know is visible in the signature of a function or procedure</li>
Replicating the parameters for documentation is error prone and does not follow the DRY principles</li>
With meaningful parameter names you need often no or only short comments - these can be placed directly after the parameters as single line comments</li>
Long descriptions can be written in the Markdown main comment</li>
</ul>
</li>
The package/function/procedure signature should be automatically included in the output as a code block for reference</li>
Output should also be Markdown - this can be easily read by humans and further processed by machines

Only documented functions/procedures should be included in the output to keep the focus on the important stuff</li>
I want to have an automatic table of contents in each generated document</li>
With the help of Pandoc</a> many formats like Word, RTF, PDF or HTML can be generated (for GitHub I only need Markdown)</li>
</ul>
</li>
</ul>
There are already several open source projects available for PL/SQL code documentation generation:</p>

PLDoc</a></li>
OraDoclet</a></li>
SchemaSpy</a></li>
Natural Docs</a></li>
PL/SQL to Markdown Documenter</a></li>
</ul>
Only the last one is able to generate Markdown docs, but the input has to be in JavaDoc. All the others can produce HTML output and often some other formats like RTF, Word or PDF. Natural Docs has a Markdown like simple text syntax, but also no Markdown output. Some of the tools can automatically document a whole schema - that is not the focus of PLOC. Most comparable is the intention of PL/SQL to Markdown  Documenter.</p>
So I had to do it by myself. I called it PLOC (PLDoc was already used, so I skipped the D…) and it is implemented in JavaScript as a npm package and currently used by me for the following two projects: PL/SQL Export Utilities</a> and Oracle Table API Generator</a>. It is very lightweight - the main code without the CLI has currently less characters as the Markdown source of this blog post. Maybe it is also useful for you. If not, have a look at the other mentioned projects.</p>
More infos about PLOC</a> itself can be found on the project page</a>.</p>
Happy coding and documenting :-)</p>
Ottmar</p>


Table API Generator v0.5.0 Available
2018-12-26T00:00:00+00:00
Short before Christmas André and I released version 0.5.0 of our PL/SQL table API generator</a>. It took a long time to come to this point. One of the reasons behind was the fact that we had to rework some internals to support feature wishes from the community. Here the most important features of the new version:</p>

Support for multicolumn primary keys</li>
Support for 12c long identifier names</li>
More control over the API features and name (10 new parameters)</li>
Insert methods:

Can have the dictionary column defaults</li>
Can return the whole row instead of the primary key</li>
</ul>
</li>
You can exclude columns (e.g. audit columns) for inserts and updates</li>
Improved template engine:

Generated code better readable because of formatted parameter lists</li>
Tables with many columns no longer breaking character limits</li>
</ul>
</li>
Special methods for testing and dummy data generation</li>
Also see the full changelog on the project site</a></li>
</ul>
Happy new year and happy coding :-)</p>
Ottmar</p>


HydeBar
2017-11-29T00:00:00+00:00
Earlier this year I wrote at the end of this post</a>:</p>

PS: In the meantime I switched complete to Markdown and Jekyll</a>, a static site generator - but this is another story for another post …</p>
</blockquote>
As always - it took a bit longer then expected. In the meantime I was fiddling around with Jekyll and modifying the default theme Minima to my needs. Then I stumbled over the JSDoc theme Minami</a> and the Liquid docs site</a> - both with nice sidebars. I wanted to have such a nice sidebar for my dev blog and began to look deeper how it was implemented. To my surprise the sidebar from the JSDoc theme Minami was complete CSS driven - cool.</p>
It took me a while to full understand how it was working. With this knowledge and the design inspirations I was able to improve the Jekyll default theme to a sidebar template. As a bonus I integrated the HTML based slideshow library Reveal.js - you can write now your blog posts and slides with markdown.</p>
The nice thing about blogging with markdown is, that you don’t need to switch your tools - you use the usual ones from the daily programming job. It goes even better when you use GitHub: You commit your changes and GitHub takes care about generating your blog content from your sources. And the result is a fast and secure static site.</p>
I had a accepted talk at the DOAG Conference + Exhibition 2017</a> in the stream “Strategy & Business Practices” about blogging for developers with Jekyll. Unfortunately short before I had an accident and broke my right upper arm :-(</p>
My colleague Markus Dötsch was taking over the presentation and I concentrated on delivering the free time project “Jekyll sidebar template”. This was the working title before I renamed it to “HydeBar” - a play with “Dr. Jekyll and Mr. Hyde” and the fact, that it is a sidebar template.</p>
Yesterday I released version 1.0.0. You can start blogging for free in five minutes with this quick start guide</a>. The online demo</a> is serving as template, documentation and showcase. The mentioned slides from the DOAG Conference</a> are included in this demo (in German, sorry…).</p>
Happy blogging :-)</p>
Ottmar</p>


Table API Generator
2016-06-10T00:00:00+00:00
Last year I needed for a project a simple table API, which is able to generate a generic change log - something like “show me for table xyz which column was changed when and what was the old value and what is the new one”. Only changed values should be stored and all tables should log into the same generic logging table. Sounds like we need a generator for this.</p>
Unfortunately there was no time in the project to create such a generator from the ground. Luckily André, a colleague of me, had started to implement such an API generator in his free time and I had only to implement the generic change log functionality - also in my free time. Our API generator open source project was born :-)</p>
Later on I needed a function to get the id from an record by the unique columns - next feature for the API generator: generate a (overloaded) function for each unique constraint on a table. Now it is a simple function call to get the id:</p>
your_var :</span>=</span> your_table_name_api.get_pk_by_unique_cols(</span></span>
    p_column_name_1 </span>=></span> 'xyz'</span>,</span></span>
    p_column_name_2 </span>=></span> 123</span>,</span></span>
    ...);</span></span></code></pre>
The API is also handling the case, when no data is found and everything without the need to look in the dictionary for the unique columns of that table or to write boilerplate code and possibly forget one column.</p>
The next thing for APEX was “For tabular forms you can’t use APIs”. We came up with the idea to generate a view for each table with an instead of trigger for inserts, updates and deletions. The trigger is simply calling the API. Now you can use APEX own “automatic row processing” and the generic logging together.</p>
In my holiday this spring I stumbled over a tweet from Phillip Salvisberg about his oddgen project</a>, a SQL Developer extension to invoke dictionary-driven code generators. I was thrilled, we can have a SQL Developer integration by writing a small PL/SQL wrapper for oddgen. I finished the integration in my holiday, I could not wait.</p>
Over the time we found out, that the generation of the thin API wrappers for our tables saves us time and makes our code more stable and readable because of the saved boilerplate code.
Business logic packages becomes invalid after table changes and API regeneration. This is good, because you can see, which API calls are now invalid. If you have simple insert or update statements you might oversee some places, where the statement is still valid for example with an added table column, but in fact does the wrong insert or update.</p>
And the last thing, what is now possible: Security - separate the data from the user interface. Simply grant execute rights on the API and business logic packages to the UI schema and disable the deletion of rows in your API packages when you don’t need it. No user will be able to drop or truncate your tables, because there are no tables in the UI schema.</p>
The generated API packages are really thin - nothing special at all. Nevertheless they are powerful time savers.</p>
You can find the sources and more informations on GitHub</a>.</p>
Happy coding :-)</p>
Ottmar</p>

Ottmar’s Notes - Generator

Create fast insert scripts

New Version of Oracle Table API Generator Available

Conclusion</h2> A big THANK YOU to all contributors - the current version of the generator would not be possible without your feedback and help.</p>

PLOC - Convert PL/SQL Code into Markdown Docs

Table API Generator v0.5.0 Available

HydeBar

Table API Generator