Creating an Initializer for SENAITE

Hello everyone. I work with Mekom Solutions a contributing organization to OpenMRS and we are currently working on the OpenMRS 3.

For OpenMRS 3 we are using SENAITE as the LIMS component. Due to the way we will package our OpenMRS 3 distributions it will not be practical to manually configure the various components and we have already built tools for most of the components that we are using (OpenMRS itself, Odoo, … etc) that allow us to initialize them at start up without requiring user interventions

We are able to initialize SENAITE by uploading an xlsx file on the web UI which works great. But we need to be able do it at startup as well.

For our use case I have made an attempt to create a python script to initialize SENAITE given a configuration file. The code below may look familiar because it is adapted from here

from plone import api as ploneapi
from Products.CMFCore.utils import getToolByName
from bika.lims import PMF
from bika.lims import logger
from bika.lims.interfaces import ISetupDataImporter
from openpyxl import load_workbook
from pkg_resources import resource_filename
from zope.component import getAdapters
import traceback

import tempfile
import transaction

try:
    from zope.component.hooks import getSite
except:
    # Plone < 4.3
    from zope.app.component.hooks import getSite


class LoadSetupData():

    def __init__(self, context):
        self.context = context
        # dependencies to resolve
        self.deferred = []

    def solve_deferred(self, deferred=None):
        unsolved = []
        deferred = deferred if deferred else self.deferred
        for d in self.deferred:
            src_obj = d['src_obj']
            src_field = src_obj.getField(d['src_field'])
            multiValued = src_field.multiValued
            src_mutator = src_field.getMutator(src_obj)
            src_accessor = src_field.getAccessor(src_obj)

            tool = getToolByName(self.context, d['dest_catalog'])
            try:
                proxies = tool(d['dest_query'])
            except:
                continue
            if len(proxies) > 0:
                obj = proxies[0].getObject()
                if multiValued:
                    value = src_accessor()
                    value.append(obj.UID())
                else:
                    value = obj.UID()
                src_mutator(value)
            else:
                unsolved.append(d)
        self.deferred = unsolved
        return len(unsolved)

    def load_data(self):
        self.dataset_project = 'bika.lims'
        self.dataset_name = 'test'
        workbook = None
        try:
            workbook = load_workbook(filename='/home/enyachoke/Code/Mekom/senaite/senaitelims/src/collective.initializer/src/collective/initializer/configuration.xlsx')  # , use_iterators=True)
        except AttributeError:
            print ""
            print traceback.format_exc()
            print "Error while loading "

        adapters = [[name, adapter]
                    for name, adapter
                    in list(getAdapters((self.context, ), ISetupDataImporter))]

        for sheetname in workbook.get_sheet_names():
            transaction.savepoint()
            ad_name = sheetname.replace(" ", "_")
            if ad_name in [a[0] for a in adapters]:
                adapter = [a[1] for a in adapters if a[0] == ad_name][0]
                adapter(self, workbook, self.dataset_project, self.dataset_name)
                adapters = [a for a in adapters if a[0] != ad_name]
            transaction.commit()
        for name, adapter in adapters:
            transaction.savepoint()
            adapter(self, workbook, self.dataset_project, self.dataset_name)
            transaction.commit()
        check = len(self.deferred)
        while len(self.deferred) > 0:
            new = self.solve_deferred()
            logger.info("solved %s of %s deferred references" % (
                check - new, check))
            if new == check:
                raise Exception("%s unsolved deferred references: %s" % (
                    len(self.deferred), self.deferred))
            check = new

        logger.info("Rebuilding bika_setup_catalog")
        bsc = getToolByName(self.context, 'bika_setup_catalog')
        bsc.clearFindAndRebuild()
        logger.info("Rebuilding bika_catalog")
        bc = getToolByName(self.context, 'bika_catalog')
        bc.clearFindAndRebuild()
        logger.info("Rebuilding bika_analysis_catalog")
        bac = getToolByName(self.context, 'bika_analysis_catalog')
        bac.clearFindAndRebuild()

        message = PMF("Changes saved.")
        self.context.plone_utils.addPortalMessage(message)
def main(app):
    loader = LoadSetupData(app.senaite)
    loader.load_data()
if "app" in locals():
    main(app)

I have tested the script with the after-installhook of collective.recipe.plonesite and it seems to work okay. My concern is that my combined experience working with Plone and SENAITE is less that 2 weeks and am not sure if my approach is any good. @Espurna @xispa any thoughts on this.

1 Like

Adding @mksd @mksrom

HI @enyachoke , the legacy “xlsx” import mechanism is old-school and not all fields are supported. In fact, not all content types are supported neither.

Best approach is to take advantage of the Generic Setup Content Structure Export/Import already in place. So basically, I would first manually populate an instance with “baseline” data (e.g. Analysis Services, Analysis profiles, etc.) and export that same data to a tarball via ZMI (see Structure Export/Import Handlers for Generic Setup by ramonski · Pull Request #1463 · senaite/senaite.core · GitHub).

Then, you can import that same tarball into fresh instances manually. For the tarball to be imported automatically, I suggest to add an additional step in your add-on’s setuphandlers.py (see Add-on Installation And Export framework: GenericSetup — Plone Documentation v5.2)

1 Like

Thanks a lot @xispa this is super helpful I will spend some more time looking into how to create the plugin. I will keep sharing my progress here.

1 Like

Hello @xispa we have created a simple plugin to initialize our SENAITE sites it reads the tar.gz file exported from ZMI and loads it into SENAITE via the runAllImportStepsFromProfile function plone.initializer/setuphandlers.py at 31d452b76d86c930b4cee3a4202debaf4a3188ae · mekomsolutions/plone.initializer · GitHub . It seems to work okay but am still not an expert in this. If you have some time you can take a look.