Generate APEX plug-ins on the command line

2021-10-13T00:00:00+00:00

Introduction</h2>
Everyone who develops plug-ins for APEX knows the problem: Once a plug-in is finished, future maintenance is usually just a matter of uploading updated JavaScript files to the plug-in and finally downloading the plug-in to make it available to others. This can be a very time consuming process when troubleshooting or implementing a new feature.</p>
So for my latest open source project, I thought about how to shorten the whole process. I already had a file watcher and a build script running anyway, which, when source files are changed, merges the entire project into a single SQL install script. The plug-in was just about updating a JavaScript file (normal and minified). The PL/SQL code for the plug-in is in a package and only referenced - this is also a good approach for performance reasons (Blog post by Daniel Hochleitner on the topic</a>).</p>

Creating a template from the existing plug-in</h2>

So I downloaded the finished plug-in once and stored it as a template in the project directory. Then I looked at how the files are stored in the SQL script of the plug-in:</p>

begin</span></span>
    wwv_flow_api.g_varchar2_table :</span>=</span> wwv_flow_api.empty_varchar2_table;</span></span>
    wwv_flow_api.g_varchar2_table(</span>1</span>) :</span>=</span> '766172206f69633d7b6f633a7b7d2c6f613a7b7d2c6c6e3a7b7d7d3b6f69632e6c6e2e6572726f723d312c6f69632e6c6e2e7761726e696e673d322c6f69632e6c6e2e696e666f3d332c6f69632e6c6e2e64656275673d342c6f69632e6c6e2e74726163'</span>;</span></span>
    wwv_flow_api.g_varchar2_table(</span>2</span>) :</span>=</span> '653d352c6f69632e746f537472696e673d66756e6374696f6e2872297b666f7228766172206f3d22222c653d303b653c722e6c656e6774683b652b2b296f2b3d30213d3d653f2220223a22222c226f626a656374223d3d747970656f6620725b655d3f6f'</span>;</span></span>
    wwv_flow_api.g_varchar2_table(</span>3</span>) :</span>=</span> '2b3d225c6e222b4a534f4e2e737472696e6769667928725b655d2c6e756c6c2c32293a6f2b3d725b655d3b72657475726e206f7d2c6f69632e696e69743d66756e6374696f6e28297b6f69632e6d6573736167653d66756e6374696f6e28722c6f2c652c'</span>;</span></span>
    /* snip */</span></span>
    wwv_flow_api.g_varchar2_table(</span>14</span>) :</span>=</span> '2e6c6e2e6572726f722c722c6e2c692e737461636b292c21317d7d3b'</span>;</span></span>
end</span>;</span></span>
/</span></span>
</span>
begin</span></span>
    wwv_flow_api.create_plugin_file (</span></span>
        p_id           </span>=></span> wwv_flow_api.id(</span>37195131994077352</span>),</span></span>
        p_plugin_id    </span>=></span> wwv_flow_api.id(</span>36295154520053378</span>),</span></span>
        p_file_name    </span>=></span> 'console.min.js'</span>,</span></span>
        p_mime_type    </span>=></span> 'application/javascript'</span>,</span></span>
        p_file_charset </span>=></span> 'utf-8'</span>,</span></span>
        p_file_content </span>=></span> wwv_flow_api.varchar2_to_blob(wwv_flow_api.g_varchar2_table) );</span></span>
end</span>;</span></span>
/</span></span></code></pre>
    Example of a file in the SQL installation script of a plug-in</figcaption>
</figure>
It looks like we have to encode the files somehow - but how? If you take a closer look at the characters, you will notice at some point that only the digits 0 to 9 and the letters a to f are used. So it seems to be a hexadecimal encoding - with 200 characters in each block.</p>
Ok, then in the template we replace the first anonymous PL/SQL block with a placeholder:</p>

    #CONSOLE_JS_FILE_MIN#</span></span>
</span>
begin</span></span>
    wwv_flow_api.create_plugin_file (</span></span>
        p_id           </span>=></span> wwv_flow_api.id(</span>37195131994077352</span>),</span></span>
        p_plugin_id    </span>=></span> wwv_flow_api.id(</span>36295154520053378</span>),</span></span>
        p_file_name    </span>=></span> 'console.min.js'</span>,</span></span>
        p_mime_type    </span>=></span> 'application/javascript'</span>,</span></span>
        p_file_charset </span>=></span> 'utf-8'</span>,</span></span>
        p_file_content </span>=></span> wwv_flow_api.varchar2_to_blob(wwv_flow_api.g_varchar2_table) );</span></span>
end</span>;</span></span>
/</span></span></code></pre>
    Example of a file in the plug-in template with placeholder</figcaption>
</figure>
Now we need some code to replace the placeholder with the converted source file.</p>
JavaScript helper functions for conversion</h2>
Since I implemented the file watcher and other scripts with Node.js/npm, it was obvious to solve this in JavaScript as well. First the needed modules and a md5 hash function:</p>
const</span> fs</span> =</span> require</span>(</span>'fs'</span>)</span>;</span></span>
const</span> UglifyJS</span> =</span> require</span>(</span>'uglify-js'</span>)</span>;</span></span>
const</span> crypto</span> =</span> require</span>(</span>'crypto'</span>)</span>;</span></span>
const</span> toMd5Hash</span> =</span> function</span> (</span>string</span>) {</span> return</span> crypto</span>.</span>createHash</span>(</span>'md5'</span>)</span>.</span>update</span>(string)</span>.</span>digest</span>(</span>'hex'</span>)</span> };</span></span>
let</span> consoleJsCode</span>,</span> minified</span>,</span> version</span>,</span> md5Hash</span>,</span> conf</span>;</span></span></code></pre>
Then we need a helper function that breaks a string into blocks of 200 characters and returns it as an array:</p>
const</span> toChunks</span> =</span> function</span> (</span>text</span>,</span> size</span>) {</span></span>
    const</span> numChunks</span> =</span> Math</span>.</span>ceil</span>(text</span>.</span>length</span> /</span> size)</span>;</span></span>
    const</span> chunks</span> =</span> new</span> Array</span>(numChunks)</span>;</span></span>
    for</span> (</span>let</span> i</span> =</span> 0</span>,</span> start</span> =</span> 0</span>;</span> i</span> <</span> numChunks</span>;</span> ++</span>i</span>,</span> start</span> +=</span> size)</span> {</span></span>
        chunks[i]</span> =</span> text</span>.</span>substr</span>(start</span>,</span> size)</span>;</span></span>
    }</span></span>
    return</span> chunks</span>;</span></span>
};</span></span></code></pre>
Finally, our actual function, which hexadecimally encodes a string (read file) and puts it into the required SQL script form:</p>
const</span> toApexPluginFile</span> =</span> function</span> (</span>text</span>) {</span></span>
    const</span> hexString</span> =</span> new</span> Buffer</span>.</span>from</span>(text)</span>.</span>toString</span>(</span>'hex'</span>)</span>;</span></span>
    const</span> chunks</span> =</span> toChunks</span>(hexString</span>,</span> 200</span>)</span>;</span></span>
    let</span> apexLoadFile</span> =</span> 'begin</span>\n</span>'</span> +</span></span>
        '  wwv_flow_api.g_varchar2_table := wwv_flow_api.empty_varchar2_table;</span>\n</span>'</span>;</span></span>
    for</span> (</span>let</span> i</span> =</span> 0</span>;</span> i</span> <</span> chunks</span>.</span>length</span>;</span> ++</span>i)</span> {</span></span>
        apexLoadFile</span> +=</span> `  wwv_flow_api.g_varchar2_table(</span>${</span>(</span>i</span> +</span> 1</span>)</span>}</span>) := '</span>${</span>chunks</span>[</span>i</span>]</span>}</span>';</span>\n</span>`</span>;</span></span>
    }</span></span>
    apexLoadFile</span> +=</span> 'end;</span>\n</span>/'</span>;</span></span>
    return</span> apexLoadFile</span>;</span></span>
};</span></span></code></pre>Build script to build the plug-in when changes are made in the source code</h2>
Everything is assembled by the build script - here is the relevant snippet:</p>

    console</span>.</span>log</span>(</span>'- build file install/apex_plugin.sql'</span>)</span>;</span></span>
consoleJsCode</span> =</span> fs</span>.</span>readFileSync</span>(</span>'sources/apex_plugin_console.js'</span>,</span> 'utf8'</span>)</span>;</span></span>
version</span> =</span> fs</span>.</span>readFileSync</span>(</span>'sources/CONSOLE.pks'</span>,</span> 'utf8'</span>)</span>.</span>match</span>(</span>/</span>c_version</span>\s</span>+</span>constant</span>.</span>*?</span>'(</span>.</span>*?</span>)'</span>/</span>)[</span>1</span>]</span>;</span></span>
md5Hash</span> =</span> toMd5Hash</span>(consoleJsCode)</span>;</span></span>
// reading the last saved version and md5Hash values as a reference for the comparison</span></span>
conf</span> =</span> JSON</span>.</span>parse</span>(fs</span>.</span>readFileSync</span>(</span>'apexplugin.json'</span>,</span> 'utf8'</span>))</span>;</span></span>
if</span> (conf</span>.</span>version</span> !==</span> version</span> ||</span> conf</span>.</span>jsFile</span>.</span>md5Hash</span> !==</span> md5Hash)</span> {</span></span>
    minified</span> =</span> UglifyJS</span>.</span>minify</span>(</span>{</span> "console.js"</span>:</span> consoleJsCode</span> }, {</span> sourceMap</span>:</span> true</span> }</span>)</span>;</span></span>
    if</span> (minified</span>.</span>error)</span> throw</span> minified</span>.</span>error</span>;</span></span>
    conf</span>.</span>version</span> =</span> version</span>;</span></span>
    conf</span>.</span>jsFile</span>.</span>md5Hash</span> =</span> md5Hash</span>;</span></span>
    conf</span>.</span>jsFile</span>.</span>version</span> +=</span> 1</span>;</span></span>
    fs</span>.</span>writeFileSync</span>(</span>'install/apex_plugin.sql'</span>,</span></span>
        '--DO NOT CHANGE THIS FILE - IT IS GENERATED WITH THE BUILD SCRIPT sources/build.js</span>\n\n</span>'</span> +</span></span>
        fs</span>.</span>readFileSync</span>(</span>'sources/apex_plugin_template.sql'</span>,</span> 'utf8'</span>)</span></span>
            .</span>replace</span>(</span>'#CONSOLE_VERSION#'</span>,</span> conf</span>.</span>version)</span></span>
            .</span>replace</span>(</span>'#FILE_VERSION#'</span>,</span> conf</span>.</span>jsFile</span>.</span>version)</span></span>
            .</span>replace</span>(</span>'#CONSOLE_JS_FILE#'</span>,</span> toApexPluginFile</span>(fs</span>.</span>readFileSync</span>(</span>'sources/apex_plugin_console.js'</span>,</span> 'utf8'</span>)))</span></span>
            .</span>replace</span>(</span>'#CONSOLE_JS_FILE_MIN#'</span>,</span> toApexPluginFile</span>(minified</span>.</span>code))</span></span>
            .</span>replace</span>(</span>'#CONSOLE_JS_FILE_MIN_MAP#'</span>,</span> toApexPluginFile</span>(minified</span>.</span>map))</span></span>
    )</span>;</span></span>
    fs</span>.</span>writeFileSync</span>(</span>'apexplugin.json'</span>,</span> JSON</span>.</span>stringify</span>(conf</span>,</span> null</span>,</span> 2</span>))</span>;</span></span>
}</span></span></code></pre>
    Example of a build script (excerpt, here the complete script</a>)</figcaption>
</figure>
It should be noted that we will also have to manage the file version itself - APEX will install it as we create it. For that purpose, we check by means of hash-function whether JavaScript source file has changed and only then we regenerate the plug-in (including minification). This way, we avoid unnecessarily regenerating the plug-in every time we change source code in the repository and thus incrementing the file version.In the build script snippet above, the current file version and hash value is stored in a JSON file (apexplugin.json</code>, last line of code) so that we can use this as a reference in the next build.</p>
Here is the relevant place in the plug-in template where the file version is entered by the build script (placeholder #FILE_VERSION#</code>, last parameter):</p>

    prompt </span>-</span> application</span>/</span>shared_components</span>/</span>plugins</span>/</span>dynamic_action</span>/</span>com_ogobrecht_console</span></span>
begin</span></span>
    wwv_flow_api.create_plugin (</span></span>
        p_id                        </span>=></span> wwv_flow_api.id(</span>36295154520053378</span>),</span></span>
        p_plugin_type               </span>=></span> 'DYNAMIC ACTION'</span>,</span></span>
        p_name                      </span>=></span> 'COM.OGOBRECHT.CONSOLE'</span>,</span></span>
        p_display_name              </span>=></span> 'Oracle Instrumentation Console'</span>,</span></span>
        p_supported_ui_types        </span>=></span> 'DESKTOP:JQM_SMARTPHONE'</span>,</span></span>
        p_api_version               </span>=></span> 2</span>,</span></span>
        p_render_function           </span>=></span> 'console.apex_plugin_render'</span>,</span></span>
        p_ajax_function             </span>=></span> 'console.apex_plugin_ajax'</span>,</span></span>
        p_substitute_attributes     </span>=></span> true,</span></span>
        p_subscribe_plugin_settings </span>=></span> true,</span></span>
        p_version_identifier        </span>=></span> '#CONSOLE_VERSION#'</span>,</span></span>
        p_about_url                 </span>=></span> 'https://github.com/ogobrecht/console'</span>,</span></span>
        p_files_version             </span>=></span> #FILE_VERSION# );</span></span>
end</span>;</span></span>
/</span></span></code></pre>
    Plug-in template with placeholders for file and plug-in version (snippet)</figcaption>
</figure>
In addition, the version number of the plug-in itself is also updated here (placeholder #CONSOLE_VERSION#</code>, third last parameter) - but this depends on the respective circumstances of the project. In my case I write the version number of the PL/SQL package CONSOLE also into the plug-in depending on it - so if the version number of the package changes, the plug-in will be rebuilt as well. But here everybody has to see what is needed in the project.</p>
When saving source code changes, the file watcher starts and triggers the build script. This then builds two installation SQL scripts together: That of the logging tool itself (not the focus of this article) and then that for the plug-in. Following this, both scripts are then installed in a dev environment. For this, of course, the default workspace stored in the plug-in and the default application number must be present in the dev environment.</p>
NPM scripts for orchestration</h2>
I now often manage build and install scripts with the node package manager npm. The reason for this is the good integration with Visual Studio Code, the operating system independence and the possibility to orchestrate the scripts a bit - ok, you have to install Node.js, but don’t you always? With one click you can either start the watch task, which does everything automatically with every save of a source file, or you can conveniently call the scripts individually. If someone else looks into the repository, it is also clearer with the npm scripts what works here and how.</p>

    
    npm script integration in VS Code</figcaption>
</figure>
It’s also clear that you don’t write usernames and passwords into the scripts here anymore - that’s already forbidden in principle when the code is checked into version control. On the screenshot you can (hopefully) see that here a wallet with the alias playground</code> is used. For those who only use the instant client and have problems setting up a wallet: How to use mkstore and orapki with Oracle Instant Client</a>.</p>
The scripts are managed in the file package.json</code> on the top level of the project directory. If you want to read more about npm scripts, you can start here:</p>

Introduction to NPM Scripts</a></li>
Why I Left Gulp and Grunt for npm Scripts</a></li>
Why npm Scripts?</a></li>
</ul>
A complete example</h2>
For those who want to see how everything can work together: I took the examples here in the article from my open source project Oracle Instrumentation Console</a>.</p>
Another way</h2>
My focus with this solution was to be able to supply a plug-in locally on the PC with new files. But there is another way: You can upload files or whole directories via script into an installed plug-in of an APEX application and then download the plug-in afterwards. For this purpose there is the Open Souce Project APEX Publish Static Files</a> by Vincent Morneau. This project is also integrated with APEX Nitro</a> which can speed up frontend development in APEX.</p>
Have fun generating plug-ins :-)</p>
Ottmar</p>

A complete example</h2>
For those who want to see how everything can work together: I took the examples here in the article from my open source project Oracle Instrumentation Console</a>.</p>

Ottmar’s Notes - npm

Generate APEX plug-ins on the command line

PLOC - Convert PL/SQL Code into Markdown Docs

Ottmar’s Notes - npm

Generate APEX plug-ins on the command line

JavaScript helper functions for conversion</h2> Since I implemented the file watcher and other scripts with Node.js/npm, it was obvious to solve this in JavaScript as well. First the needed modules and a md5 hash function:</p>

A complete example</h2> For those who want to see how everything can work together: I took the examples here in the article from my open source project Oracle Instrumentation Console</a>.</p>

PLOC - Convert PL/SQL Code into Markdown Docs

A complete example</h2>
For those who want to see how everything can work together: I took the examples here in the article from my open source project Oracle Instrumentation Console</a>.</p>