ScripterNG

This article is part of the Scripts series.

ScripterNG is a new plugin to provide scripting facilities in Python and Javascript (actually Ecmascript via QtScript).

This plugin is still in development. As of september 2014, work to be done is

• get it to work again (probably a matter of porting it to qt5 and python3)
• find a way to cleanly document it
• complete the missing functions
• remove the parts that are not correctly working (like the internal editor) or clean it up

Notes :

• The changes in the code breaking it are (mostly) small issues.
• Some fixes have allready been merged in GIT branch : https://github.com/aoloe/scribus-plugin-scripter

---

Attention: This information is obsolete and needs to be updated The new scripter is now part of the trunk branch in svn. The plugin was based on the myplugin example code which is also documented in the Plugin-Howto.

Quickstart

• get Scribus from svn trunk (the Qt4 port is needed)
• get a snapshot from http://henning.cco-ev.de/scribus/scripterng.tgz or use BazarrNG:

  bzr branch http://henning.cco-ev.de/scribus/scripterng

• extract the files into the plugin directory (scribus/plugins/scripterng)
• add ADD_SUBDIRECTORY(scripterng) in scribus/plugins/CMakeLists.txt
• (perhaps disable scriptplugin for testing)
• install python-dev, python-qt4-dev, perhaps sip-dev if you haven't already
• run cmake, make, make install
• go to scribus/plugins/scripterng/scribuspyqt
• run cmake, make and copy *.py and *.so to INSTALLPREFIX/lib/scribus/plugins/scripterng (sorry this is a bug)
• start scribus
• create/open document
• menu Extras -> ScripterNG
• test and see messages on console
• send patches - thanks :)
• retry, fix thinks I forgot to mention here..

Development

About the files

• cmake/modules/*
  • Some extra macros from PyKDE to find SIP and PyQt.
  • They are used in scribuspyqt/. There seems to exist some conflicts with other existing macros.
  • So I had to compile scribuspyqt alone
  • I am not sure if they work well on Windows.
• pythonize.{cpp,h}
  • This is a nice small interface to use Python in C++
• scripterng.{cpp,h}
  • hooks into Extras menu
  • low level plugin stuff (like in myplugin)
• scripterngimpl.{cpp,h}
  • parent is QApplication::instance()
  • sets objectName to ScripterNG (only named objects are exposed)
  • loads Python and runs scripterng.py on startup
  • implements example function: aboutScripterNG()
  • object is magically available as Application.ScripterNG
• scribuspyqt/scribuspyqt.{cpp,h}
  • compiles to libscribuspyqt.so
  • provides additional code to wrap without sip
  • invokeMethod using variants (to call unwrapped objects)
• scribuspyqt/scribuspyqt.sip
  • sip interface to make invokeMethod available to Python
  • creates python module _scribuspyqt.sip
  • links against libscribuspyqt.so
• scribuspyqt/scribuspyqt.py
  • import this module instead of _scribuspyqt directly
  • PyQtObject can be used to wrap any QObject
    • slots can be called like methods
    • full property support
• scribuspyqt/scripterng.py
  • called at startup from plugin
  • looks for application instance and makes it available
  • will be used to hook into Scribus with PyQt, to provide menus to load scripts etc
• scribuspyqt/pyqtscript.py
  • prove of concept: activates QtScript from Python
  • no speed loss
  • mix Python and QtScript
• scribuspyqt/compat.py
  • start of a compatibility layer
  • provides same api as the "old" scripter

Making functions available to ScripterNG

Here is an example:

class CoreModule : QObject {
   CoreModule();
   public slots:
      void quit(bool ask_save);
      int answer();
}
CoreModule::CoreModule() : QObject(
          QApplication::instance()->findChild("ScripterNG")) {
  setObjectName("Core");
}
void CoreModule::quit(bool ask_save) {
  if (ask_save() && confirm() .. ) {
     qDebug("Bye");
     QApplication::instance()->quit();
  }
}
int CoreModule::answer() {
  return 42;
}
CoreModule *m_core = new CoreModule();

Now you can do

>> print Application.Core.answer()
42
>> Application.Core.quit(true)
Bye

API brainstorming

Petr wrote: I'd like to have some logical "object-related" structure for it. Something like e.g.:

 scribus.Document().pages[0].items[0].text()

Henning: I had similar thoughts. Instead of methods we can use the properties in Python directly:

scribus.CurrentDocument.Pages[0].Items[0].Text

This will call the get- and set-methods automatically.

Perhaps we should design an API which is similar to the API used in KOffice:

• KWord
• KSpread
• Krita
• SuperKaramba
• Kopete

See also the wiki page on public C++ API

• Here are some thoughts I compiled with Umbrello:
  • Signals can be useful to allow to register for actions like 'new page added' or 'preferences changed'
  • What about widgets? GUI-integration e.g. in the menu or toolbar or in a dialog?
  • Probably it makes sense to provide specialized classes to be able to write import/export scripts (reader/writer), document-templates, dockers/QDockWidget's, etc.
  • Execution of a script should be defered as long as possible. This makes a lot of sense if e.g. scripts are used for import, export, event-driven things, etc. + a simple py_initialize and 'import PyQt4' may already block the GUI for seconds depending on the hardware + it scales bad with lot of scripts that may never got executed by the user during a session.

Comparison with Kross

• ScripterNG and Kross both use the introspection features of Qt. The script language would not see any difference between called by ScripterNG or Kross. So if we start writing the C++ API now. We could switch to Kross later and could still use this API.
  • true, but it may start to get difficult if the C++ API earns additional code to work around e.g. reference-counting probs (not that they are needed, depends on how the backends is done). Re basic types / QVariant-handling, I strongly suggest to write unittests for this! Specialy it's needed to don't only test basic types like double, int and QString but also extended types like QRect and QPoint since they are handled different internaly by Qt4. I also would recommed to rethink if there is a way to use such a C++ API also for dbus. This is not much work taken into account that QDBusAbstractAdaptor's are QObject's too, but would allow to also support the same rich interface via dbus but may need additional work re reference-counting aka handling object lifetime the right(TM) way :) [sebsauer]
    • Actually I trust PyQt doing the reference counting for me. The scribuspyqt is only a very thin layer on top of it. Nevertheless some tests look useful and I will write a few. [Henning]
      • Well, such a unittest-suite may not only be useful for a specific backend but also for the C++ API itself. Think of a way to execute a small app with "./runDaTest" and to have verified that the whole published API still works even after a bigger change of the internal/private functionality. Regression's are an evil thing and it may rather difficult to discover them or even more worse, let users discover them. For sure imho it would also make sense to have such tests for a specific backend, but main consumer is still the solution-independend C++ API. If such tests are even written in e.g. python, then also a/the backend itself got tested what means 2 in 1 :) I guess if the goal is to keep such an API stable for a long time, it makes quit sense to invest the additional time to have proper regression-tests for the exposed API (ok, maybe that's also a result of my own very good experiences with test-driven XP-development but nevermind it can't harm :) [sebsauer]
• Kross btw is great but it currently still depends on kdelibs. ScripterNG is available now. Of course it isn't that mature but it depends on mature technologies (PyQt, sip, pythonize).
  • re kdelibs-dependency; in kdelibs/kross/core open the files interpreter.cpp+manager.cpp and s/klibloader/qlibrary and in action.cpp kmimetype is used to fetch the icons for python+ruby+etc (so, could be easy made optional). Everything else is Qt4 only + all existing bindings (ruby, python, kjs+kjsembed, java and falcon) are Qt4 only already since quit some time. Unfortunately we lost the timeframe to get anything into kdelibs or even out of it for 4.0. While 4.1 may provide the next possibility to do so, it may also an idea to copy some of the code atm within kdelibs/kross/core to the backends themself and remove that way any remaining link-dependency to kdelibs/kross/core. This can be probably done any time depending on the freez-state of kdebindings. [sebsauer]
• One main difference is that ScripterNG tries to do most stuff in Python. So it can be more flexible and development should be faster. Any sensible speed loss by this approach is not expected.
• Kross also only has support for KJSEmbed (part of kdelibs) and not for QtScript (see http://akademy.kde.org/conference/slides/kjsembed-and-qtscript.pdf).
  • this is void (aka pre Qt4.2 knowledge) since QtScript-support is already done, but both ways; Kross can be used in QtScript and QtScript in Kross. No way to get this in now since kdelibs is under hard freez. But if you are interested, I could provide a patch-set from my local repo :) [sebsauer]
• there is a more important issue yet; I still wait for my WindowsXp-license to arrive and as long as that didn't happened I can't say for sure that the backends do work out-of-box on Windows(C) right now even if they should. Anyway, let's hope those bureaucratic within my university (who I'll earn my license from) may passed soon. [sebsauer]
• there are also quit some advantages yet. To name the probably most interesting one is the possibility that we are able to connect the "C++ API modules" on the fly. As example see here the sample how SuperKaramba accesses KSpread through Kross. On the other hand, I didn't test this with Qt-only apps yet and would assume that there are probably some problems related to missing instances of KComponent's that arn't easy to sort out :-( Another advantage is to share code. So, there is no need for additional packaging for the python/ruby/whatever-dependency + all scripting-backends are optional only anyways what is imho important for those distributors that deliver small solutions like embedded or mobile stuff. Same goes here for the PyQt+sip dependency. Kross does allow full access to them but does not depend on them, they are optional too and a script is able to check that at runtime + fallback to e.g. tkinter if it likes. [sebsauer]

• To make this clear: I really like Kross and I am happy that KDE now contains such a great technology. The only reason I started this plugin was that I wanted to learn something about sip and more about C++. And I could not wait until someone ports Kross to a Scribus plugin. For me this task looked too big. But Kross convinced me that using the Qt object system is the way to go instead of writing bindings for everything. The scribuspyqt-part of the plugin could be an interim solution which could later be replaced by Kross. In scripterngimpl.cpp only a few lines have to be changed. [Henning]
  • Probably my replies where a bit offensive. Well, to make it clear; in no way I guess Kross is the only or even best solution. One of the big advantages the FOSS-world has is, that we don't stick with one solution forever or are limited to one single solution. That's why I also did named dbus above since that's even a theird use-case (or forth if native C++ plugins would use the same interface). I guess the only important thing here is to try to keep the C++ API free from any to much specific solution (aka, try to prevent any kind of python or ruby or whatever related code in the C++ API themself). There is just no way to know now what will happen next few years but its clear, that such a API may need a lot of mainting-work over the years. So, its much easier if such a API does the only job it is designed for; provide a stable interface and leave implementation aka backend details up to optional plugins. From the very beginning one of the main goals of Kross was to be optional and to keep the bload out of the apps themself by moving it into plugins. So, lot of text, short conclusion: I am all for having more options then only one of the ScripterNG, dbus, QtScript, Kross, [put in another thing] solutions. I guess if it's possible to keep that as top-priority during designing+implementation of the stable C++ API, Scribus will earn a very flexible, easy to maintain and ready for next few years solution. So, I would be happy to be allowed to help there in the process however to achieve that goal (p.s. we did it the same way in SuperKaramba; there we have the "old" python-backend as well as Kross and it's possible to switch between them easy what was really a help during the development-process) plus every application that uses Kross so far does use it via a plugin. So, neither KSpread nor xyz does depend on it. My very own believing is anyway, that next few years we will see a move back from the 'we have desktop-power, so let's consume it' Eclipse-like bloadware (sorry, just an example, I could also name Borland2007 here) to smaller and less performance-eating devices+software (embedded, mobile, handhelds, green-IT, but also longer running laptops, etc.) what makes it even more important to be able to optional remove parts of a software depending on the use-cases without patching the sources (seems we went the same way in KOffice btw where nearly everything are plugins now). Also sharing the same interfaces for different backends doesn't only make documentation easier but also to translate e.g. a python-script to a C++ plugin. [sebsauer]