https://wiki.scribus.net/wiki/api.php?action=feedcontributions&user=Ringerc&feedformat=atomScribus Wiki - User contributions [en]2024-03-28T18:55:57ZUser contributionsMediaWiki 1.39.1https://wiki.scribus.net/wiki/index.php?title=Public_C%2B%2B_API&diff=15190Public C++ API2007-04-07T19:51:20Z<p>Ringerc: /* Public C++ API */</p>
<hr />
<div>{{Scripting Index}}<br />
[[Category:Scribus_development]]<br />
<br />
== Public C++ API ==<br />
<br />
Scribus's internal interfaces are in constant flux as code is improved and refactored. They're also sometimes complex, hard to use, and often underdocumented. Often performance takes priority over easy understanding for the new user, and some have a considerable amount of historical cruft.<br />
<br />
For all these reasons, we would benefit signifiantly from providing a new public C++ API to Scribus's functionality. This would be a collection of wrappers around the core code that:<br />
<br />
* Hide internal-only structure and complexity<br />
* Abstracted some functionality into simpler, but perhaps slower, interfaces<br />
* Handle ownership issues related to exposing the API to refcounting or GC'd languages with:<br />
** Handles or proxy objects for important internal objects<br />
** Factory functions rather than direct ctor use<br />
* Hides direct use of pointers to structures behind indexes into structures, names, proxy objects, or other abstractions.<br />
* Logically groups functionality, often consolidating a number of objects into one<br />
** Needs interfaces to:<br />
*** Text system<br />
*** Pages<br />
*** PageItems<br />
*** Document manipulation (sizes, export, page management)<br />
*** Application-wide functions (open doc etc)<br />
*** GUI (menus, toolbars, etc)<br />
** See current scripter for an outline of some of the needed functionality.<br />
* Provides important hooks for extending the GUI (exposes menu manager, key toolbars, action manager, etc)<br />
* Exposes signals that can be connected to when a script/plugin wants to be informed of important events<br />
<br />
=== Design Beginnings ===<br />
<br />
We need to start, I think, by (a) looking at how others have done it (especially KDE apps supporting Kross), and (b) writing down everything we want the plug-in API to enable people to do. The existing scripter will serve as the beginnings of a guide to what's needed.<br />
<br />
Once we know what we want to initially expose, we collect functionality into logical groups or entities based on what it applies to or acts on (works on the entire app vs works on the document vs works on a text pageitem, etc). That gives the beginnings of the structure we'd need the API to have as far as users of it are concerned.<br />
<br />
=== Details ===<br />
<br />
==== Object Lifetimes & Memory Management ====<br />
<br />
One thing I am pretty clear on is how to handle object lifetime issues.<br />
A script or plug-in may wish to use objects that existed before it was<br />
loaded, or wish to create objects that will exist after it is unloaded.<br />
It should be able to do this without explicitly managing object<br />
lifetimes or performing manual memory management - especially for<br />
interfaces to languages that handle memory differently (using a gc, refcounting, etc). I think the best<br />
way to handle this is the use of proxy objects, or handles, in cases where a 1:1<br />
shadowing of an internal object is required.<br />
<br />
Deleting a proxy object ( a handle ) should not affect the behaviour or<br />
lifetime of the shadowed internal object. Nor should creating one. The<br />
handle should be able to tell when the referenced object is<br />
invalidated/destroyed and flag its self as invalid, so further<br />
operations on it throw an exception. For shadowed QObjects this is easily accomplished by connecting the proxy to a signal such as the automatic destroyed() signal emitted by a QObject as it's deleted, or a custom signal indicating invalidation for more complex cases. In other words, a handle to an<br />
object should work much like a weak smart pointer does, but should also<br />
provide wrappers for useful API for the internal object. The behaviour<br />
described above matches closely to how SIP works, and it's far from<br />
perfect, but probably as good as you'll get when faced with an<br />
environment where a user might be manipulating the same objects your<br />
script/plugin is.<br />
<br />
==== Units ====<br />
<br />
Another thing I think is critically important is that all API calls<br />
should use the internal units. Experience with the scripter has shown<br />
that supporting user-set units in code is the quick way to scripts that<br />
behave inconsistently based on different user settings, bugs,<br />
"interesting" floating point rounding issues, and general insanity. All<br />
API calls should use `pt'. No exceptions. It should be possible to use<br />
the API to query for the units the user wants to see so scripts can<br />
convert displayed units, and input data, to the right units, and we<br />
should probably eventually expose some widgets and input dialogs that<br />
handle units automatically. All API calls should take measurements in<br />
pt, however, and return measurements in pt. API users can use the<br />
exported units enums to convert from pt to other units, or maybe<br />
conversion functions - not sure which is the better interface.<br />
<br />
Having multiple units supported in the API leads to bugs where the plugin/script is tested and modified by a user using one set of units, and consequently malfunctions when used by other users. It also leads to a lot of unnecessary unit conversions, possibly causing a loss of precision. Units only need conversion on display to the user or input from the user, and should only be converted then. We can provide tools, as mentioned above, to make it easier for script and plugin writers to take input and display data in the users' preferred units, but should stick to our internal units for anything not directly involving the user.<br />
<br />
=== Possible Benefits ===<br />
<br />
If done well, this public API may let us actually hide much of the current innards entirely from scripts and plug-ins. They should see only the wrapper API headers, and should need to link only to wrapper API symbols. We should be able to reduce the size of the exported symbol table (improving binary size and link times), more easily expose functionality to new languages like QtScript, make it easier to write highly functional plug-ins and scripts, etc. Handily, we could also provide a more _stable_ interface to plug-ins and scripts, so fewer core changes would require incompatible changes in scripts or plug-ins.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Scripter_Security&diff=15178Scripter Security2007-04-07T17:48:05Z<p>Ringerc: </p>
<hr />
<div>{{Scripting Index}}<br />
<br />
== Scripter Security Discussion ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
=== Answer to CR===<br />
<br />
I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
=== Reply (CR) ===<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is extremely unlikely to work. PyQt in particular gives the programmer pretty much free reign to do whatever they want. <br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking. Permitting users useful access to PyQt and the Qt introspection system is going to let them get at what they want. Qt is full of methods that access the file system, network, and more, and I'm not sure there's a realistic way to restrict access to them. I'd love to be proved wrong, of course.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps a limited automation scripting function can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you may not be familiar with in depth is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt "extension script" use etc) or in a subinterpreter. This causes its own problems, and I wrote it as a dirty hack until we could find a sane way to solve the problem.<br />
<br />
I think the sub-interprter problems are another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become fully capable and full-featured plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' I really liked the work you did before on the extension manager and your OpenClipArt work, and if you remember I spent some time helping to get PyQt to actually work in Scribus so they'd be more viable. My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15177Improve scripting and its experience in Scribus2007-04-07T17:47:31Z<p>Ringerc: </p>
<hr />
<div>{{Scripting Index}}<br />
<br />
== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed [[public C++ API]] for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see [[Scripter Security]] ]<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository [CR: Great idea, especially if combined with signing of scripts for integrity etc]<br />
* scripts look and behave as native parts of Scribus<br />
<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
[CR: discussion moved to [[Talk:Improve_scripting_and_its_experience_in_Scribus|The talk page]] for this article.]<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* Bindings for the preferences manager [CR]<br />
* Bindings for the menu manager [CR]<br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]<br />
<br />
<br />
<br />
----<br />
<br />
<br />
= Excerpt from submitted abstract and application to GSoC 2007 =<br />
<br />
== Rationale == <br />
Scribus shows that serious desktop publishing is possible with free software. Scribus has many powerful features which are commonly mentioned in articles, e.g. the high quality PDF export and the color management. <br />
But the potential of the scripting interface is mostly underestimated. The Scripter plug-in is included in the Scribus main distribution together with some example scripts. However it does not get the attention it deserves in my point of view. <br />
Because of that I want to show what is possible with the Python scripting interface in Scribus. <br />
<br />
== Overview ==<br />
Other open-source projects like Firefox, Blender, Inkscape, OpenOffice or Gimp have shown that bindings to scripting languages result in many new features and contribute largely to the success of the application. People without knowledge of C/C++ but some Python-, JavaScript- or Perl-skills can easily write import-/export-filters, create wizards, write plug-ins or realize something nobody else has thought about before. The distributed scripts can be used without recompiling and even without knowledge of the scripting language.<br />
<br />
Scripts can help to automate tasks and can fit the special needs of a user. Also it is possible to test new ideas. Languages like Python are great for prototyping. And sometimes it makes more sense to stay with Python instead of rewriting everything in a compiled language.<br />
<br />
As Scribus already provides a good basis with its Scripter plug-in I want to improve it and its usability. To show the power of the scripting interface I want to write some small and some larger example scripts (resp. extension scripts). My idea is that Scribus needs a script-/extension-manager like Firefox has for themes and add-ons. Like Firefox a user can browse the scripts on a website and install them by just clicking a link. So my task will also include building a small web application to provide a repository of scripts.<br />
<br />
So the project consists of three tasks which are mutually dependent. <br />
1. The script-manager based on PyQt and its QTextBrowser<br />
2. The example scripts and extensions. Planed are an OpenClipArt browser and a simple spell-checker.<br />
3. The repository. Either a CGI or a script which generates static pages<br />
<br />
At each task I will determine which functionally is missing and how these sub-projects can be linked together. This will show what needs to be done in the scripting interface. Currently scripts do not look like native Scribus components because e.g. they cannot be connected to menu-/toolbar-entries or context-menus. There is also no standard way to install scripts to make them automatically available. These things need to be addressed. <br />
Some implementation issues will be discussed with the Scribus team, e.g. about security (already partly done), integration or future directions.<br />
Currently there are thoughts in the Scribus team about creating new automatic bindings. Because of that I will do two things:<br />
1. I will try to keep my code modular and flexible. It should be relatively easy to port the script-/extension-manager to the new bindings when available.<br />
2. I will evaluate different binding generators (sip, swig, boost, smoke) and make tests to see which will be the best path to go (probably sip with a separate C++ API)<br />
<br />
<br />
<br />
== Benefits ==<br />
1. Firefox showed that people liked the flexibility to install add-ons. My project will bring this feature to Scribus in the hope that people will enjoy Scribus even more and it will attract new users.<br />
2. The script interface and its experience will improve which will hopefully attract new script writers.<br />
3. Scribus gets a repository for scripts anyone can easily access. I would be willing to host it after the GSoC.<br />
4. Some C++ code is not needed anymore and could probably be removed, e.g. the script console. I already wrote a better replacement in PyQt.<br />
<br />
<br />
== Deliverables ==<br />
* a script manager<br />
* example scripts<br />
* a script repository<br />
* documentation for the script writers</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Public_C%2B%2B_API&diff=15176Public C++ API2007-04-07T17:46:55Z<p>Ringerc: /* Public C++ API */</p>
<hr />
<div>{{Scripting Index}}<br />
<br />
== Public C++ API ==<br />
<br />
Scribus's internal interfaces are in constant flux as code is improved and refactored. They're also sometimes complex, hard to use, and often underdocumented. Often performance takes priority over easy understanding for the new user, and some have a considerable amount of historical cruft.<br />
<br />
For all these reasons, we would benefit signifiantly from providing a new public C++ API to Scribus's functionality. This would be a collection of wrappers around the core code that:<br />
<br />
* Hide internal-only structure and complexity<br />
* Abstracted some functionality into simpler, but perhaps slower, interfaces<br />
* Handle ownership issues related to exposing the API to refcounting or GC'd languages with:<br />
** Handles or proxy objects for important internal objects<br />
** Factory functions rather than direct ctor use<br />
* Hides direct use of pointers to structures behind indexes into structures, names, proxy objects, or other abstractions.<br />
* Logically groups functionality, often consolidating a number of objects into one<br />
** Needs interfaces to:<br />
*** Text system<br />
*** Pages<br />
*** PageItems<br />
*** Document manipulation (sizes, export, page management)<br />
*** Application-wide functions (open doc etc)<br />
*** GUI (menus, toolbars, etc)<br />
** See current scripter for an outline of some of the needed functionality.<br />
* Provides important hooks for extending the GUI (exposes menu manager, key toolbars, action manager, etc)<br />
* Exposes signals that can be connected to when a script/plugin wants to be informed of important events<br />
<br />
We need to start, I think, by (a) looking at how others have done it (especially KDE apps supporting Kross), and (b) writing down everything we want the plug-in API to enable people to do. The existing scripter will serve as the beginnings of a guide to what's needed.<br />
<br />
Once we know what we want to initially expose, we collect functionality into logical groups or entities based on what it applies to or acts on (works on the entire app vs works on the document vs works on a text pageitem, etc). That gives the beginnings of the structure we'd need the API to have as far as users of it are concerned.<br />
<br />
If done well, this public API may let us actually hide much of the current innards entirely from scripts and plug-ins. They should see only the wrapper API headers, and should need to link only to wrapper API symbols. We should be able to reduce the size of the exported symbol table (improving binary size and link times), more easily expose functionality to new languages like QtScript, make it easier to write highly functional plug-ins and scripts, etc. Handily, we could also provide a more _stable_ interface to plug-ins and scripts, so fewer core changes would require incompatible changes in scripts or plug-ins.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Plugin_Architecture&diff=15175Plugin Architecture2007-04-07T17:46:27Z<p>Ringerc: </p>
<hr />
<div>{{Scripting Index}}<br />
<br />
Scribus can be extended and automated using C++ plugins. Plugins are dynamic loaded libraries in "so" format.<br />
<br />
An introduction to writing Scribus plugins is available here: [[http://docs.scribus.net/index.php?lang=en&sm=pluginhowto&page=plugin_howto http://docs.scribus.net/index.php?lang=en&sm=pluginhowto&page=plugin_howto]]. At present (3 Feb '05) there's not a lot of documentation available for the object model. It's said to be in Doxygen format in the source code, although I can't find it.<br />
<br />
There is also a separate api for formatted text importer plugins. More information on this can be found from here:<br />
[[http://docs.scribus.net/index.php?lang=en&sm=gettexthowto&page=gettexthowto http://docs.scribus.net/index.php?lang=en&sm=gettexthowto&page=gettexthowto]]<br />
<br />
Scribus also provides a Python [[Scripter API]], which is generally easier to use, more mature and better documented than the C++ API. The internal C++ API is often updated and is not compatible release-to-release. Work on a [[public C++ API]] that's simpler, more stable, easier to use, and exposes more functionality for scripting languages is being considered.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Plugin_Architecture&diff=15174Plugin Architecture2007-04-07T17:46:06Z<p>Ringerc: </p>
<hr />
<div>{{Scripting Index}}<br />
<br />
Scribus can be extended and automated using C++ plugins. Plugins are dynamic loaded libraries in "so" format.<br />
<br />
An introduction to writing Scribus plugins is available here: [[http://docs.scribus.net/index.php?lang=en&sm=pluginhowto&page=plugin_howto http://docs.scribus.net/index.php?lang=en&sm=pluginhowto&page=plugin_howto]]. At present (3 Feb '05) there's not a lot of documentation available for the object model. It's said to be in Doxygen format in the source code, although I can't find it.<br />
<br />
There is also a separate api for formatted text importer plugins. More information on this can be found from here:<br />
[[http://docs.scribus.net/index.php?lang=en&sm=gettexthowto&page=gettexthowto http://docs.scribus.net/index.php?lang=en&sm=gettexthowto&page=gettexthowto]]<br />
<br />
Scribus also provides a Python [[Scripter API]], which is generally easier to use, more mature and better documented than the C++ API. The internal C++ API is often updated and is not compatible release-to-release. Work on a [public C++ API] that's simpler, more stable, easier to use, and exposes more functionality for scripting languages is being considered.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Public_C%2B%2B_API&diff=15173Public C++ API2007-04-07T17:40:58Z<p>Ringerc: /* Public C++ API */</p>
<hr />
<div>== Public C++ API ==<br />
<br />
Scribus's internal interfaces are in constant flux as code is improved and refactored. They're also sometimes complex, hard to use, and often underdocumented. Often performance takes priority over easy understanding for the new user, and some have a considerable amount of historical cruft.<br />
<br />
For all these reasons, we would benefit signifiantly from providing a new public C++ API to Scribus's functionality. This would be a collection of wrappers around the core code that:<br />
<br />
* Hide internal-only structure and complexity<br />
* Abstracted some functionality into simpler, but perhaps slower, interfaces<br />
* Handle ownership issues related to exposing the API to refcounting or GC'd languages with:<br />
** Handles or proxy objects for important internal objects<br />
** Factory functions rather than direct ctor use<br />
* Hides direct use of pointers to structures behind indexes into structures, names, proxy objects, or other abstractions.<br />
* Logically groups functionality, often consolidating a number of objects into one<br />
** Needs interfaces to:<br />
*** Text system<br />
*** Pages<br />
*** PageItems<br />
*** Document manipulation (sizes, export, page management)<br />
*** Application-wide functions (open doc etc)<br />
*** GUI (menus, toolbars, etc)<br />
** See current scripter for an outline of some of the needed functionality.<br />
* Provides important hooks for extending the GUI (exposes menu manager, key toolbars, action manager, etc)<br />
* Exposes signals that can be connected to when a script/plugin wants to be informed of important events<br />
<br />
We need to start, I think, by (a) looking at how others have done it (especially KDE apps supporting Kross), and (b) writing down everything we want the plug-in API to enable people to do. The existing scripter will serve as the beginnings of a guide to what's needed.<br />
<br />
Once we know what we want to initially expose, we collect functionality into logical groups or entities based on what it applies to or acts on (works on the entire app vs works on the document vs works on a text pageitem, etc). That gives the beginnings of the structure we'd need the API to have as far as users of it are concerned.<br />
<br />
If done well, this public API may let us actually hide much of the current innards entirely from scripts and plug-ins. They should see only the wrapper API headers, and should need to link only to wrapper API symbols. We should be able to reduce the size of the exported symbol table (improving binary size and link times), more easily expose functionality to new languages like QtScript, make it easier to write highly functional plug-ins and scripts, etc. Handily, we could also provide a more _stable_ interface to plug-ins and scripts, so fewer core changes would require incompatible changes in scripts or plug-ins.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Public_C%2B%2B_API&diff=15172Public C++ API2007-04-07T17:35:31Z<p>Ringerc: /* Public C++ API */</p>
<hr />
<div>== Public C++ API ==<br />
<br />
Scribus's internal interfaces are in constant flux as code is improved and refactored. They're also sometimes complex, hard to use, and often underdocumented. Often performance takes priority over easy understanding for the new user, and some have a considerable amount of historical cruft.<br />
<br />
For all these reasons, we would benefit signifiantly from providing a new public C++ API to Scribus's functionality. This would be a collection of wrappers around the core code that:<br />
<br />
* Hide internal-only structure and complexity<br />
* Abstracted some functionality into simpler, but perhaps slower, interfaces<br />
* Handle ownership issues related to exposing the API to refcounting or GC'd languages with:<br />
** Handles or proxy objects for important internal objects<br />
** Factory functions rather than direct ctor use<br />
* Hides direct use of pointers to structures behind indexes into structures, names, proxy objects, or other abstractions.<br />
* Logically groups functionality, often consolidating a number of objects into one<br />
** Needs interfaces to:<br />
*** Text system<br />
*** Pages<br />
*** PageItems<br />
*** Document manipulation (sizes, export, page management)<br />
*** Application-wide functions (open doc etc)<br />
*** GUI (menus, toolbars, etc)<br />
* Provides important hooks for extending the GUI (exposes menu manager, key toolbars, action manager, etc)<br />
* Exposes signals that can be connected to when a script/plugin wants to be informed of important events</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Public_C%2B%2B_API&diff=15171Public C++ API2007-04-07T17:33:23Z<p>Ringerc: </p>
<hr />
<div>== Public C++ API ==<br />
<br />
Scribus's internal interfaces are in constant flux as code is improved and refactored. They're also sometimes complex, hard to use, and often underdocumented. Often performance takes priority over easy understanding for the new user, and some have a considerable amount of historical cruft.<br />
<br />
For all these reasons, we would benefit signifiantly from providing a new public C++ API to Scribus's functionality. This would be a collection of wrappers around the core code that:<br />
<br />
- Hid internal-only structure and complexity<br />
- Abstracted some functionality into simpler, but perhaps slower, interfaces<br />
- Handle ownership issues related to exposing the API to refcounting or GC'd languages with:<br />
- Handles or proxy objects for important internal objects<br />
- Factory functions rather than direct ctor use<br />
- Hides direct use of pointers to structures behind indexes into structures, names, proxy objects, or other abstractions.<br />
- Logically groups functionality, often consolidating a number of objects into one<br />
- Needs interfaces to:<br />
- Text system<br />
- Pages<br />
- PageItems<br />
- Document manipulation (sizes, export, page management)<br />
- Application-wide functions (open doc etc)<br />
- GUI (menus, toolbars, etc)<br />
- Provides important hooks for extending the GUI (exposes menu manager, key toolbars, action manager, etc)<br />
- Exposes signals that can be connected to when a script/plugin wants to be informed of important events</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15170Improve scripting and its experience in Scribus2007-04-07T17:24:40Z<p>Ringerc: /* Current problems */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed [[public C++ API]] for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see [[Scripter Security]] ]<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository [CR: Great idea, especially if combined with signing of scripts for integrity etc]<br />
* scripts look and behave as native parts of Scribus<br />
<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
[CR: discussion moved to [[Talk:Improve_scripting_and_its_experience_in_Scribus|The talk page]] for this article.]<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* Bindings for the preferences manager [CR]<br />
* Bindings for the menu manager [CR]<br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]<br />
<br />
<br />
<br />
----<br />
<br />
<br />
= Excerpt from submitted abstract and application to GSoC 2007 =<br />
<br />
== Rationale == <br />
Scribus shows that serious desktop publishing is possible with free software. Scribus has many powerful features which are commonly mentioned in articles, e.g. the high quality PDF export and the color management. <br />
But the potential of the scripting interface is mostly underestimated. The Scripter plug-in is included in the Scribus main distribution together with some example scripts. However it does not get the attention it deserves in my point of view. <br />
Because of that I want to show what is possible with the Python scripting interface in Scribus. <br />
<br />
== Overview ==<br />
Other open-source projects like Firefox, Blender, Inkscape, OpenOffice or Gimp have shown that bindings to scripting languages result in many new features and contribute largely to the success of the application. People without knowledge of C/C++ but some Python-, JavaScript- or Perl-skills can easily write import-/export-filters, create wizards, write plug-ins or realize something nobody else has thought about before. The distributed scripts can be used without recompiling and even without knowledge of the scripting language.<br />
<br />
Scripts can help to automate tasks and can fit the special needs of a user. Also it is possible to test new ideas. Languages like Python are great for prototyping. And sometimes it makes more sense to stay with Python instead of rewriting everything in a compiled language.<br />
<br />
As Scribus already provides a good basis with its Scripter plug-in I want to improve it and its usability. To show the power of the scripting interface I want to write some small and some larger example scripts (resp. extension scripts). My idea is that Scribus needs a script-/extension-manager like Firefox has for themes and add-ons. Like Firefox a user can browse the scripts on a website and install them by just clicking a link. So my task will also include building a small web application to provide a repository of scripts.<br />
<br />
So the project consists of three tasks which are mutually dependent. <br />
1. The script-manager based on PyQt and its QTextBrowser<br />
2. The example scripts and extensions. Planed are an OpenClipArt browser and a simple spell-checker.<br />
3. The repository. Either a CGI or a script which generates static pages<br />
<br />
At each task I will determine which functionally is missing and how these sub-projects can be linked together. This will show what needs to be done in the scripting interface. Currently scripts do not look like native Scribus components because e.g. they cannot be connected to menu-/toolbar-entries or context-menus. There is also no standard way to install scripts to make them automatically available. These things need to be addressed. <br />
Some implementation issues will be discussed with the Scribus team, e.g. about security (already partly done), integration or future directions.<br />
Currently there are thoughts in the Scribus team about creating new automatic bindings. Because of that I will do two things:<br />
1. I will try to keep my code modular and flexible. It should be relatively easy to port the script-/extension-manager to the new bindings when available.<br />
2. I will evaluate different binding generators (sip, swig, boost, smoke) and make tests to see which will be the best path to go (probably sip with a separate C++ API)<br />
<br />
<br />
<br />
== Benefits ==<br />
1. Firefox showed that people liked the flexibility to install add-ons. My project will bring this feature to Scribus in the hope that people will enjoy Scribus even more and it will attract new users.<br />
2. The script interface and its experience will improve which will hopefully attract new script writers.<br />
3. Scribus gets a repository for scripts anyone can easily access. I would be willing to host it after the GSoC.<br />
4. Some C++ code is not needed anymore and could probably be removed, e.g. the script console. I already wrote a better replacement in PyQt.<br />
<br />
<br />
== Deliverables ==<br />
* a script manager<br />
* example scripts<br />
* a script repository<br />
* documentation for the script writers</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Talk:Improve_scripting_and_its_experience_in_Scribus&diff=15142Talk:Improve scripting and its experience in Scribus2007-04-02T20:02:13Z<p>Ringerc: /* Discussion */</p>
<hr />
<div>== Discussion ==<br />
<br />
=== ... on the final proposal ===<br />
(Added by CR after the proposal was uploaded:)<br />
<br />
The proposal that was submitted seems sound, with sensible goals that are achievable starting with the current code base. It seems vastly better than what the initial outline on the wiki described, and should be amenable to being adapted to a new and more complete scripting interface if anyone ever finds the time to write one.<br />
<br />
=== On the earlier proposal ideas/outline ===<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project. I also think you have some great ideas with regards to making scripts easier to find, providing sensible and consistent ways to install them, etc. Don't let me discourage you ... I'm not trying to, I'm just trying to point out potential issues with the ''approach'' rather than goals.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose Scribus's functionality to scripts and plug-ins. I spent quite a long time maintaining the scripter alongside Petr, so I am all to familiar with how much breakage there is from version to version and how hard it can be to keep up.<br />
<br />
I'm also very strongly in agreement with you in terms of the desirablity of hooks into Scribus for use by scripts and plugins. I've been thinking about that for a while; see extension script page referenced below. <br />
<br />
Some articles on the wiki that may be of note: [[Scripter Security]], [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
<br />
CR: Do you have a good idea of what form the new API will take? [[User:Malex|Malex]] 00:51, 31 March 2007 (CEST)<br />
<br />
Malex: Honestly, no. API design like this is *HARD*, especially when going for something that makes sense as Scribus advances. The existing scripter, and the recognised limitations in it, provide some ideas about some of the things people need to be able to do through such an API, though, as does what plugins are currently doing. What we need to do is look at how these tasks, and ones that people want to be able to do but can't do, are being done right now, and how we can help make them easier and saner. Some internal changes will be needed too, such as finishing the page item separation to the point where factory-created page items that can be supplied by plug-ins are viable. Unfortunately, what's needed here is to sit down and spend a LOT of time examining use cases, thinking very hard about usability of the interfaces and the simplicity-vs-flexibility trade off, etc. Lots and lots of time. That's why nobody, myself included, has jumped to do it yet, though both Petr and I see it as a necessary step before we can really make much progress improving the scripter or making plugins easier to write. --[[User:Ringerc|Ringerc]] 22:02, 2 April 2007 (CEST)</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Scripter_Security&diff=15030Scripter Security2007-03-24T18:02:23Z<p>Ringerc: /* Reply (CR) */</p>
<hr />
<div>== Scripter Security Discussion ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
=== Answer to CR===<br />
<br />
I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
=== Reply (CR) ===<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is extremely unlikely to work. PyQt in particular gives the programmer pretty much free reign to do whatever they want. <br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking. Permitting users useful access to PyQt and the Qt introspection system is going to let them get at what they want. Qt is full of methods that access the file system, network, and more, and I'm not sure there's a realistic way to restrict access to them. I'd love to be proved wrong, of course.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps a limited automation scripting function can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you may not be familiar with in depth is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt "extension script" use etc) or in a subinterpreter. This causes its own problems, and I wrote it as a dirty hack until we could find a sane way to solve the problem.<br />
<br />
I think the sub-interprter problems are another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become fully capable and full-featured plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' I really liked the work you did before on the extension manager and your OpenClipArt work, and if you remember I spent some time helping to get PyQt to actually work in Scribus so they'd be more viable. My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Talk:Improve_scripting_and_its_experience_in_Scribus&diff=15029Talk:Improve scripting and its experience in Scribus2007-03-24T17:47:11Z<p>Ringerc: /* Discussion */</p>
<hr />
<div>== Discussion ==<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project. I also think you have some great ideas with regards to making scripts easier to find, providing sensible and consistent ways to install them, etc. Don't let me discourage you ... I'm not trying to, I'm just trying to point out potential issues with the ''approach'' rather than goals.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose Scribus's functionality to scripts and plug-ins. I spent quite a long time maintaining the scripter alongside Petr, so I am all to familiar with how much breakage there is from version to version and how hard it can be to keep up.<br />
<br />
I'm also very strongly in agreement with you in terms of the desirablity of hooks into Scribus for use by scripts and plugins. I've been thinking about that for a while; see extension script page referenced below. <br />
<br />
Some articles on the wiki that may be of note: [[Scripter Security]], [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15028Improve scripting and its experience in Scribus2007-03-24T17:46:32Z<p>Ringerc: /* Discussion */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see [[Scripter Security]] ]<br />
<br />
<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository [CR: Great idea, especially if combined with signing of scripts for integrity etc]<br />
* scripts look and behave as native parts of Scribus<br />
<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
[CR: discussion moved to [[Talk:Improve_scripting_and_its_experience_in_Scribus|The talk page]] for this article.]<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* Bindings for the preferences manager [CR]<br />
* Bindings for the menu manager [CR]<br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Talk:Improve_scripting_and_its_experience_in_Scribus&diff=15027Talk:Improve scripting and its experience in Scribus2007-03-24T17:45:38Z<p>Ringerc: </p>
<hr />
<div>== Discussion ==<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project. I also think you have some great ideas with regards to making scripts easier to find, providing sensible and consistent ways to install them, etc. Don't let me discourage you ... I'm not trying to, I'm just trying to point out potential issues with the ''approach'' rather than goals.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose Scribus's functionality to scripts and plug-ins. I spent quite a long time maintaining the scripter alongside Petr, so I am all to familiar with how much breakage there is from version to version and how hard it can be to keep up.<br />
<br />
I'm also very strongly in agreement with you in terms of the desirablity of hooks into Scribus for use by scripts and plugins. I've been thinking about that for a while; see extension script page referenced below. <br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15026Improve scripting and its experience in Scribus2007-03-24T17:41:33Z<p>Ringerc: /* Implementation details */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see [[Scripter Security]] ]<br />
<br />
<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository [CR: Great idea, especially if combined with signing of scripts for integrity etc]<br />
* scripts look and behave as native parts of Scribus<br />
<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
== Discussion ==<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project. I also think you have some great ideas with regards to making scripts easier to find, providing sensible and consistent ways to install them, etc. Don't let me discourage you ... I'm not trying to, I'm just trying to point out potential issues with the ''approach'' rather than goals.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose Scribus's functionality to scripts and plug-ins. I spent quite a long time maintaining the scripter alongside Petr, so I am all to familiar with how much breakage there is from version to version and how hard it can be to keep up.<br />
<br />
I'm also very strongly in agreement with you in terms of the desirablity of hooks into Scribus for use by scripts and plugins. I've been thinking about that for a while; see extension script page referenced below. <br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* Bindings for the preferences manager [CR]<br />
* Bindings for the menu manager [CR]<br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15025Improve scripting and its experience in Scribus2007-03-24T17:40:04Z<p>Ringerc: /* Discussion */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see [[Scripter Security]] ]<br />
<br />
<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository [CR: Great idea, especially if combined with signing of scripts for integrity etc]<br />
* scripts look and behave as native parts of Scribus<br />
<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
== Discussion ==<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project. I also think you have some great ideas with regards to making scripts easier to find, providing sensible and consistent ways to install them, etc. Don't let me discourage you ... I'm not trying to, I'm just trying to point out potential issues with the ''approach'' rather than goals.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose Scribus's functionality to scripts and plug-ins. I spent quite a long time maintaining the scripter alongside Petr, so I am all to familiar with how much breakage there is from version to version and how hard it can be to keep up.<br />
<br />
I'm also very strongly in agreement with you in terms of the desirablity of hooks into Scribus for use by scripts and plugins. I've been thinking about that for a while; see extension script page referenced below. <br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Extension_script_discussion&diff=15024Extension script discussion2007-03-24T17:38:40Z<p>Ringerc: /* Discussion of extension script matters */</p>
<hr />
<div>{{Scripting Index}}<br />
<br />
== Discussion of extension script matters ==<br />
<br />
Any discussion of extension script plans, issues, etc should probably happen here. The [[Known Scripter Issues]] and [[Cool experimental PyQt stuff]] pages may also be of interest.<br />
<br />
==== Open Issues ====<br />
<br />
These areas need work, consideration, and discussion. The more (considered and useful) feedback and ideas, the better.<br />
<br />
* Extension scripts<br />
** How to make it easy for the user to differentiate extension and normal scripts<br />
** How best to load extension scripts (I'm now tending toward importing as modules then running a function in the module)<br />
** How best to explain what an extension script is vs a normal script<br />
** Should I impose an import hook to prevent loading of PyQt in normal scripts (since it'll crash Scribus)?<br />
** Worth looking into sip bindings for the Scribus-specific GUI classes?<br />
** Provide a standardised way to delay initialisation of scripts until app loading finishes<br />
* Scripter core<br />
** Property/object based interface is working in 1.3. Do we care about doing automatic unit translation, or is it better to just let the user say '1*in' in arguments? I favour the latter.<br />
** A basic object-based interface is now possible. How should it look?<br />
* Dependencies<br />
** What issues do folks have with dependencies such as sip or libboost for scripter? I don't want to make any such deps required for basic function, but it might be possible to do some cool optional functionality with them.<br />
* Hooks<br />
<br />
=== Hooks ===<br />
<br />
For listen-only hooks, Scribus doesn't need to do much more than emit a signal that scripts and plugins can connect(...) to. However, the event loop won't be running when the hook gets called, and there may also be certain things it's unsafe to do because scribus just isn't expecting them. For that reason, in some cases it might be better to use posted events rather than signals.<br />
<br />
Where signal-based hooks would be really useful would be where the script can affect how things run. For example, the script might want to run some additional checks when PDF export is requested, and might display a dialog letting the user cancel the operation if the script finds problems (think: "Text box `story' contains data that's out of date according to the CMS"). For this, the script would connect(...) to a signal, and would be able to call a method like abortCurrentOperation() in the core code to indicate the user request to cancel. We can't just have the signal return a flag since, being one-to-many calls, they can't have return values.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15023Improve scripting and its experience in Scribus2007-03-24T17:26:03Z<p>Ringerc: /* For the user */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see [[Scripter Security]] ]<br />
<br />
<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository [CR: Great idea, especially if combined with signing of scripts for integrity etc]<br />
* scripts look and behave as native parts of Scribus<br />
<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
== Discussion ==<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project. I also think you have some great ideas with regards to making scripts easier to find, providing sensible and consistent ways to install them, etc. Don't let me discourage you ... I'm not trying to, I'm just trying to point out potential issues with the ''approach'' rather than goals.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose Scribus's functionality to scripts and plug-ins. I spent quite a long time maintaining the scripter alongside Petr, so I am all to familiar with how much breakage there is from version to version and how hard it can be to keep up.<br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15022Improve scripting and its experience in Scribus2007-03-24T17:25:06Z<p>Ringerc: /* Security */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see [[Scripter Security]] ]<br />
<br />
<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
== Discussion ==<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project. I also think you have some great ideas with regards to making scripts easier to find, providing sensible and consistent ways to install them, etc. Don't let me discourage you ... I'm not trying to, I'm just trying to point out potential issues with the ''approach'' rather than goals.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose Scribus's functionality to scripts and plug-ins. I spent quite a long time maintaining the scripter alongside Petr, so I am all to familiar with how much breakage there is from version to version and how hard it can be to keep up.<br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Scripter_Security&diff=15021Scripter Security2007-03-24T17:24:44Z<p>Ringerc: </p>
<hr />
<div>== Scripter Security Discussion ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
=== Answer to CR===<br />
<br />
I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
=== Reply (CR) ===<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is raving lunacy... somewhat akin to giving a loaded handgun to a toddler, but saying it's safe because the safety catch is on. PyQt in particular gives the programmer pretty much free reign to do whatever they want.<br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps this can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm still not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you're probably not familiar with is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt use etc) or in a subinterpreter. This causes its own problems.<br />
<br />
I think that's another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' I really liked the work you did before on the extension manager and your OpenClipArt work, and if you remember I spent some time helping to get PyQt to actually work in Scribus so they'd be more viable. My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15020Improve scripting and its experience in Scribus2007-03-24T17:24:22Z<p>Ringerc: /* Current problems */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see [[Scripter Security]] ]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
=== Answer to CR===<br />
<br />
I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
=== Reply (CR) ===<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is raving lunacy... somewhat akin to giving a loaded handgun to a toddler, but saying it's safe because the safety catch is on. PyQt in particular gives the programmer pretty much free reign to do whatever they want.<br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps this can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm still not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you're probably not familiar with is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt use etc) or in a subinterpreter. This causes its own problems.<br />
<br />
I think that's another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' I really liked the work you did before on the extension manager and your OpenClipArt work, and if you remember I spent some time helping to get PyQt to actually work in Scribus so they'd be more viable. My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
== Discussion ==<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project. I also think you have some great ideas with regards to making scripts easier to find, providing sensible and consistent ways to install them, etc. Don't let me discourage you ... I'm not trying to, I'm just trying to point out potential issues with the ''approach'' rather than goals.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose Scribus's functionality to scripts and plug-ins. I spent quite a long time maintaining the scripter alongside Petr, so I am all to familiar with how much breakage there is from version to version and how hard it can be to keep up.<br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15019Improve scripting and its experience in Scribus2007-03-24T17:23:50Z<p>Ringerc: /* Discussion */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see below]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
=== Answer to CR===<br />
<br />
I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
=== Reply (CR) ===<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is raving lunacy... somewhat akin to giving a loaded handgun to a toddler, but saying it's safe because the safety catch is on. PyQt in particular gives the programmer pretty much free reign to do whatever they want.<br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps this can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm still not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you're probably not familiar with is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt use etc) or in a subinterpreter. This causes its own problems.<br />
<br />
I think that's another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' I really liked the work you did before on the extension manager and your OpenClipArt work, and if you remember I spent some time helping to get PyQt to actually work in Scribus so they'd be more viable. My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
== Discussion ==<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project. I also think you have some great ideas with regards to making scripts easier to find, providing sensible and consistent ways to install them, etc. Don't let me discourage you ... I'm not trying to, I'm just trying to point out potential issues with the ''approach'' rather than goals.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose Scribus's functionality to scripts and plug-ins. I spent quite a long time maintaining the scripter alongside Petr, so I am all to familiar with how much breakage there is from version to version and how hard it can be to keep up.<br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15018Improve scripting and its experience in Scribus2007-03-24T17:18:47Z<p>Ringerc: /* Security */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see below]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
=== Answer to CR===<br />
<br />
I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
=== Reply (CR) ===<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is raving lunacy... somewhat akin to giving a loaded handgun to a toddler, but saying it's safe because the safety catch is on. PyQt in particular gives the programmer pretty much free reign to do whatever they want.<br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps this can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm still not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you're probably not familiar with is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt use etc) or in a subinterpreter. This causes its own problems.<br />
<br />
I think that's another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' I really liked the work you did before on the extension manager and your OpenClipArt work, and if you remember I spent some time helping to get PyQt to actually work in Scribus so they'd be more viable. My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
== Discussion ==<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose to scripts and plug-ins.<br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15017Improve scripting and its experience in Scribus2007-03-24T17:15:45Z<p>Ringerc: /* My idea */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see below]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
=== Answer to CR===<br />
<br />
I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
=== Reply (CR) ===<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is raving lunacy... somewhat akin to giving a loaded handgun to a toddler, but saying it's safe because the safety catch is on. PyQt in particular gives the programmer pretty much free reign to do whatever they want.<br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps this can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm still not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you're probably not familiar with is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt use etc) or in a subinterpreter. This causes its own problems.<br />
<br />
I think that's another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.<br />
<br />
* [ in fact I've done much of this already, as has Henning. It's possible within limits to extend Scribus's GUI from Python and it's quite practical to write additional PyQt UI in floating palettes, dialogs, etc. ]<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
== Discussion ==<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose to scripts and plug-ins.<br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15016Improve scripting and its experience in Scribus2007-03-24T17:15:17Z<p>Ringerc: /* My idea */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see below]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
=== Answer to CR===<br />
<br />
I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
=== Reply (CR) ===<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is raving lunacy... somewhat akin to giving a loaded handgun to a toddler, but saying it's safe because the safety catch is on. PyQt in particular gives the programmer pretty much free reign to do whatever they want.<br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps this can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm still not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you're probably not familiar with is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt use etc) or in a subinterpreter. This causes its own problems.<br />
<br />
I think that's another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.<br />
<br />
* [ in fact I've done much of this already, as has Henning. It's possible within limits to extend Scribus's GUI from Python and it's quite practical to write additional PyQt UI in floating palettes, dialogs, etc. ]<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access to every internal Scribus method really isn't ever going to be realistic, nor would it be useful. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. For example, a plugin or script should not be aware of, or have to care about, the details of how text is represented in Scribus's internal data structures ... it should just get access to a somewhat higher level API for text manipulation. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That, in my opinion, would be a really GREAT GSoC project.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose to scripts and plug-ins.<br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15015Improve scripting and its experience in Scribus2007-03-24T17:11:21Z<p>Ringerc: /* My idea */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see below]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
=== Answer to CR===<br />
<br />
I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
=== Reply (CR) ===<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is raving lunacy... somewhat akin to giving a loaded handgun to a toddler, but saying it's safe because the safety catch is on. PyQt in particular gives the programmer pretty much free reign to do whatever they want.<br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps this can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm still not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you're probably not familiar with is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt use etc) or in a subinterpreter. This causes its own problems.<br />
<br />
I think that's another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.<br />
<br />
* [ in fact I've done much of this already, as has Henning. It's possible within limits to extend Scribus's GUI from Python and it's quite practical to write additional PyQt UI in floating palettes, dialogs, etc. ]<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
''CR: reply'': Full access really isn't ever going to be realistic. There's a lot of internal functionality in Scribus that neither scripts nor plug-ins should ever see or be able to see. Much of it also changes quite quickly, so exposing it would cause major compatibility problems. What I suggest is that an important initial step before any sort of serious effort to expose more of Scribus to scripts is to provide a sane C++ API that hides much of these internal details behind more stable and somewhat simplified interfaces. These interfaces could then be wrapped using SIP for use by scripts and plug-ins.<br />
<br />
That's not to say that permitting full introspection via PyQt (tagging everything with object names) is inherently a bad idea, but it will lead to scripts that'll only work for one particular version of Scribus. They'll sometimes break even on patch releases. It's useful as an interim step, I agree, but it's not something I'd ever seriously rely on. Wrapping Scribus's internal objects with SIP isn't much better, either, since they're hardly API stable. IMO we _REALLY_ need a more stable C++ API to expose to scripts and plug-ins.<br />
<br />
Some articles on the wiki that may be of note: [[Known_Scripter_Issues]], [[Experimental_PyQt_projects]] and [[Extension_script_discussion]] .<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15014Improve scripting and its experience in Scribus2007-03-24T17:01:49Z<p>Ringerc: /* Security */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see below]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
=== Answer to CR===<br />
<br />
I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
=== Reply (CR) ===<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is raving lunacy... somewhat akin to giving a loaded handgun to a toddler, but saying it's safe because the safety catch is on. PyQt in particular gives the programmer pretty much free reign to do whatever they want.<br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps this can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm still not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you're probably not familiar with is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt use etc) or in a subinterpreter. This causes its own problems.<br />
<br />
I think that's another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.<br />
<br />
* [ in fact I've done much of this already, as has Henning. It's possible within limits to extend Scribus's GUI from Python and it's quite practical to write additional PyQt UI in floating palettes, dialogs, etc. ]<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15013Improve scripting and its experience in Scribus2007-03-24T17:00:34Z<p>Ringerc: qu</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see below]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
''Answer to CR'': I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
''CR: continuation''<br />
<br />
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.<br />
<br />
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.<br />
<br />
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is raving lunacy... somewhat akin to giving a loaded handgun to a toddler, but saying it's safe because the safety catch is on. PyQt in particular gives the programmer pretty much free reign to do whatever they want.<br />
<br />
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.<br />
<br />
The downside of the above is that making it secure is almost certainly wishful thinking.<br />
<br />
I think a limited and more secure scripting interface is a '''completely different problem''' to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps this can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm still not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.<br />
<br />
Another issue you're probably not familiar with is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt use etc) or in a subinterpreter. This causes its own problems.<br />
<br />
I think that's another reason to pick a different tool that's designed from the beginning for the restricted scripting uses we want. Yes, this does mean that there'd be two different scripting languages - a "power scripting and extension writing" environment and an "application automation" environment. Essentially, Python scripts would become plug-ins, with a more suitable language used for basic scripting tasks. In a way I think that's a good thing, especially if we used JavaScript (QSA). Many apps, such as the Adobe suite, use JavaScript/ECMAScript for automation and scripting, so this fits in much better with common practice. In the OSS world, Perl and LISP-variant languages are more common for embedding ... and I for one wouldn't inflict either of those on a Scribus user.<br />
<br />
'''I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it.''' My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.<br />
<br />
* [ in fact I've done much of this already, as has Henning. It's possible within limits to extend Scribus's GUI from Python and it's quite practical to write additional PyQt UI in floating palettes, dialogs, etc. ]<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=15012Improve scripting and its experience in Scribus2007-03-24T16:30:41Z<p>Ringerc: /* Security */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see below]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
''Answer to CR'': I am aware of the [http://www.amk.ca/python/howto/rexec/rexec.html problems with restricted execution] and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:<br />
* Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]<br />
* Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]<br />
* Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]<br />
* Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]<br />
* Evaluate [http://mail.python.org/pipermail/python-dev/2003-August/037791.html RestrictedPython from Zope] which proves to work in Zope and is not related to the old rpython/Bastion modules.<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
''Answer to CR'': Of course it would be great to have direct access to all objects and their methods. And this would be a great final goal. Currently with the introspection features of Qt (children, queryList, allWidgets, ..) you can only get the base classes, so you get QMainWindow and not ScribusMainWindow. Wrapping everything with SIP is a large task and I think it should be done bit by bit when something is needed. I am currently looking what needs to be done to wrap Scribus classes with sip.<br />
As an interim solution it would already helpful for finding an widget if most widgets in Scribus would have an internal name which can be set with QObject.setName or with the constructor. <br />
Then already a lot is possible. Hooking into the GUI, responding to signals etc can be done.<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=SoC2007_ideas&diff=14906SoC2007 ideas2007-03-21T05:40:05Z<p>Ringerc: /* Text frame usability enhancement */</p>
<hr />
<div>[[Category:Scribus_development]] [[Category:GSOC]]<br />
=Google Summer of Code 2007 projects ideas=<br />
<br />
Here is a list of projects that could be done during Google Summer of Code 2007.<br />
<br />
== Scribus Team Ideas ==<br />
<br />
=== Imposition===<br />
<br />
Write an imposition (booklet printing) plugin for Scribus.<br />
<br />
==== Background Information ====<br />
<br />
* [http://en.wikipedia.org/wiki/Imposition Wikipedia]<br />
* [http://desktoppub.about.com/od/imposition/a/imposition_2.htm About.com]<br />
* [http://desktoppub.about.com/od/imposition/Imposition.htm Preparing for imposition (about.com)]<br />
* [http://www.digitalmediadesigner.com/2001/10_oct/features/alap2dn.htm How it is done in Adobe Indesign]<br />
* [http://www.digitalmediadesigner.com/2001/10_oct/features/alap4.htm How it is done in Quark Express]<br />
* Two examples of how it is done in CorelDraw: [http://www.epiloglaser.com/tl_impositionvar.htm 1], [http://www.corel.com/servlet/Satellite?pagename=Corel3/Section/Display&sid=1047024315119&gid=1047024354238&cid=1047024935656&trkid=cgf0505tip1 2]<br />
<br />
=== Text merge ===<br />
<br />
Write a text merge plugin for Scribus. This would allow scribus to read data records from a table and use it for text content or filenames for image frames. It could also provide a mode to print labels etc.<br />
<br />
:There is a [http://bugs.scribus.net/view.php?id=990 mail merge RFE]. One of Czech student is working on it. It looks like it will turn into common usable text merge from various data sources (CSV, XML, db...) --[[User:Subik|Subik]] 13:05, 7 March 2007 (CET)<br />
<br />
=== End-to-end publishing solution ===<br />
<br />
This is a meta-project that could be distilled into one or more workable GSoC projects.<br />
<br />
[[Draft of end-to-end publishing solution|End-to-end publishing solution]]<br />
<br />
[[GSoC 2007 Publishing Solution Task Plan]] - This is the result of a discussion in #scribus on 2007-03-16.<br />
<br />
=== LiveCD ===<br />
[[Live DTP CD|Live DTP CD or DVD]] &ndash; this draft/project is 2 years old and should be reviewed first, but there is definitely a need for such a solution.<br />
<br />
It should be doable within the GSoC timeframe by creating a CDD (Custom Debian Distribution).<br />
<br />
=== Text frame usability enhancement ===<br />
<br />
Improve on-page text editing usability and performance (ringerc).<br />
<br />
Currently, editing text on the page is slow and clumsy, and suffers from some limitations that make it less useful than it could be. Scribus's usability would be considerably enhanced by improving this, so the story editor isn't needed for minor editing tasks. In particular, tweaking layout and formatting would be much easier if on-page text editing could be improved.<br />
<br />
There's existing work related to this area, especially the new text system work, so you wouldn't just want to jump into this. Getting in touch with Andreas Vox might be the best starting point, to find out what is already done or will be done, and what won't be touched by the new text system work.<br />
<br />
=== Enhance inline objects ===<br />
<br />
Enhance inline objects (ringerc).<br />
<br />
Inline objects are supported in Scribus, but not easily discovered. Support for positioning and sizing them is poor, and they aren't editable once they've been pasted into a run of text. Considerable improvement is needed in formatting and editing them to make them more useful.<br />
<br />
=== Merge Field Support ===<br />
<br />
Implement true "merge" style field support / variable text /conditional text a-la FrameMaker (ringerc).<br />
<br />
In technical writing, automated mailing, and certain other uses it is desirable to dynamically alter sections of the text.<br />
<br />
The most common need is to substitute certain values into `fields' in the text, as in a mail merge. Supporting such fields would be very useful, especially as Scribus's support for external data sources improves and it becomes more suitable for automated use.<br />
<br />
The second major need is to handle regions of text that can be hidden or shown based on a variable. This can be useful with technical writing. Consider: maintaining manuals for with-wifi and without-wifi variants of a DSL router in a single document by turning off wifi-related sections when exporting without-wifi version. It's also useful for mail merge, as sometimes you want to include or exclude sections of a merge document based on parameters of the record's information. For example, users paying with different methods might need different information, or maybe you want to tell "premium" customers something else. Whatever.<br />
<br />
Adobe's FrameMaker has supported these features for a long time, but few other tools handle anything like it - let alone tools like Scribus with decent layout capabilities. Many people who need this sort of thing currently use PDF generation libraries to produce their documents using languages like Java.<br />
<br />
=== Document-embedded images ===<br />
<br />
Support loading images from alternate sources. You should handle existing filesystem links and docuement-embedded images ( XML CDATA/PCDATA sections ), and leave room for things like a URL handler for HTTP images.<br />
<br />
=== Externally linked text ===<br />
<br />
Building on the concept of an XML editor (see below), implement support for using text in externally linked files (as we currently do for images). Linking should support normal file system URLs, but leave room for handling WebDAV etc in the future. There is a clear role for this in working with content management systems such as in an end-to-end publishing system.<br />
<br />
== Tales from the Crypt, a.k.a. Ideas from the Bug Tracker ==<br />
<br />
=== XML Editor ===<br />
*[http://bugs.scribus.net/view.php?id=1394 #1394 XML Editor]<br />
* Related to [http://bugs.scribus.net/view.php?id=5190 #5190]<br />
* Related to [http://bugs.scribus.net/view.php?id=2108 #2108]<br />
* Related to [http://bugs.scribus.net/view.php?id=1791 #1791]<br />
<br />
It could be practical to have an on/off toggle in the text editor between Wysiwyg and some kind of XML. One click you have the classical way to type text, Another one, formating disappear, XML tags appear like this :<br />
<br />
<nowiki><br />
--------------------------------------<br />
<font name="Bistream Vera Sans MS" size="10"><br />
blah blah blahalahb blah blah<br />
</font><br />
<justify>blih blih blih blah blah</justify><br />
<style name="paragraph"><br />
ploplpoplpoplploplpo<br />
frudubulubruih :)<br />
</style><br />
--------------------------------------<br />
</nowiki><br />
<br />
notice the difference between the (ugly) formating tags at the beginning, and the descriptive style tags, wich then define the look of the text.<br />
<br />
The ability to expose the internal XML for text objects might also be rather interesting for the Python interface. (ringerc)<br />
<br />
The approaches of Inkscape and InDesign are worth looking at. (C schaefer)<br />
<br />
Applicants could choose between two different approaches here:<br />
<br />
# Write an enhancement to the Story Editor to enable users to edit the XML content of '''text frames'''.<br />
# Write an XML editor to edit the whole Scribus document (as Inkscape does with SVG)<br />
<br />
<br />
<br />
Inkscape's re-architecturing effort: [http://wiki.inkscape.org/wiki/index.php/SubsystemRearchitecture Subsystem Architecture]<br />
<br />
=== OpenOffice import/export extension for the Story Editor ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=5190 Text export feature: ODT or other]<br />
<br />
Start with the OO text import plugin and work out a plugin that would export valid Open Document Format (ODF) xml for re-import into OpenOffice.<br />
<br />
Scribus-specific elements and attributes would need to be stripped off, of course.<br />
<br />
=== Multiple Search and Replace ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1105 #1105 Search and Replace for the whole document]<br />
<br />
Allow users to do multiple search/replace action at once in a document. For each string we’d be able to access all settings available right now for just one text string. This will add computing power to Scribus, from the UI.<br />
<br />
=== Individual snappable baseline grid lines ===<br />
*[http://bugs.scribus.net/view.php?id=2899 #2899 Snap items to baseline grid]<br />
<br />
[[Baseline Grid Enhancements]] - Discussion page.<br />
<br />
=== Snap guides to locked objects ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=3343 Snap guides to items]<br />
<br />
Instead of letting objects snap to guides, sometimes the reverse is needed, namely letting guides snap to (locked) objects. It would be nice to have this, e.g. by dragging guides while pressing the shift key. The reverse mode should be visible in the cursor (other colour or something similar).<br />
<br />
=== Latex Text Frames ===<br />
*[http://bugs.scribus.net/view.php?id=128 #128 TeX rendering engine]<br />
<br />
I think using LaTeX for setting text (including math, footnotes, bibliography and indices) and Scribus for the layout would make a very good match. But that means you want Scribus to control the position, size and shape of the textframes, and having access to Scribus's colors and fonts wouldn't be bad, either. <br />
<br />
I think it would be better to make it easy to use the same fonts for LaTeX frames and non-LaTeX frames. AFAIK font installation was/is a bigger issue in LaTeX than it ever were in Scribus. If there is a way to automate it, we should take it.<br />
<br />
What I thought of was that Scribus should only provide an interface to LaTeX in the first step: Write LaTeX commands in the SE and let LaTeX do the rendering. At this stage, Scribus style options etc. aren't involved at all.<br />
<br />
Still, LaTeX needs to know the pagesize; or you have to use preview.sty to get pages cropped to the bounding box.<br />
<br />
<br />
Hm, I guess you could do similar thinks with my solution. You'd just have a separate set of text options for LaTeX. If you take a look at fntguide.dvi from the LaTeX docs you'll understand why. There might be some Scribus options which could be translated to LaTeX parameters, but I doubt there will ever be a 1:1 correspondence. I'd be happy if I manage to find a way to translate Scribus' fonts into appropiate LaTeX font load commands and find a suitable math font for each...<br />
<br />
=== Auto-change quotes based on language settings ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=2021 #2021 - Auto-change quotes to typographical quotes depending on language settings]<br />
<br />
I'm looking for a way to automatically change quotes (", ') to typographical quotes (unicode 201c/201e etc.). Would be great if I could use the text filters to do something like: "replace every other occurence of " with unicode 201e, the rest with 201c"<br />
<br />
=== Add spellchecking to Story Editor ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=11 Add Spellcheck functionality to Story Editor]<br />
<br />
Using one of the standard libraries existing on Linux (aspell, ispell, MySpell), or possibly even use Enchant [http://www.abiword.org/enchant/]<br />
<br />
=== Automatic Widow/Orphan Control ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=132 132 - Widow/orphan control]<br />
<br />
=== Add a Clone tool as in Inkscape ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1704 Implement a 'clone' tool a la Inkscape]<br />
<br />
If this matched the featureset of Inkscape's clone tools, this would match the features requested in bugs 922*, 923 and 924, and as an extra comment in bug 344.<br />
<br />
=== PDF Export enhancement for PDF embedding ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1699 Enhance PDF exporter to enabled embedding PDF within PDF for PDF 1.4+]<br />
<br />
Enhance PDF exporter to enabled embedding PDF within PDF for PDF 1.4+ per roadmap<br />
<br />
=== Implement JDF support ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=2460 Implement JDF (Job Definition Format)]<br />
<br />
[http://www.cip4.org/documents/jdf_specifications/JDF1.3.pdf JDF] seems to be the coming standard for job tickets. It may be patent encumbered in the US and countries with so-called FTAs with the US, but scribus should definitely offer the possibility to use the format somehow to be considered a "serious" application.<br />
<br />
In particular, Quark has added support for using JDF as a source of preflight information. It's something I've wondered about for a while, and ended up talking to some of the MS Publisher folks about too. It turns out they're not interested in in-app preflight (despite how much easer it'd make life for publishing shops that have to take .pub jobs from users who can't read a job spec to save their lives).<br />
<br />
If implementation of JDF is too large a task to implement during SoC, applicants could consider implementing one of the subsets, like the [http://www.heidelberg.com/www/html/en/content/articles/prinect/ppf Print Production Format (PPF)] or the [http://www.adobe.com/supportservice/custsupport/NOTES/86e2.htm Portable Job Ticket Format (PJTF)].<br />
<br />
=== Rich Text Format import plugin ===<br />
<br />
Implement an RTF importer based on the existing plugins for other formats. (Investigate [http://sourceforge.net/projects/librtf librtf]'s features.)<br />
<br />
=== WordPerfect Format import plugin ===<br />
<br />
Implement a WordPerfect importer based on the existing plugins for other formats. (Investigate [http://sourceforge.net/projects/libwpd/ libwpd]'s features.)<br />
<br />
===Other Import Filters===<br />
<br />
Applicants could also decide to write import plugins for other file formats. Please have look at the [http://wiki.scribus.net/index.php/File_formats_that_should_be_supported_by_Scribus_%28wish_list%29 list of desired import filters] and the [http://wiki.scribus.net/index.php/New_Colours_Format list of colour formats].<br />
<br />
== Blue Sky Ideas ==<br />
<br />
Present the title of your idea here and link to the wiki page where you elaborate on it.<br />
<br />
=== twexter ===<br />
<br />
[http://twext.com/faq.html twext] texts provide [http://sk.com.br/sk-krash.html comprehensible input] for language learners, and can integrate will many softwares (including [http://meta.wikimedia.org/wiki/Summer_of_Code_2007/proposals MediaWiki] and/or the [http://wiki.laptop.org/go/WiXi wiki] for the $100 laptop ).. why not a twexter plugin for Scribus?<br />
<br />
=== Improve scripting ===<br />
<br />
[[Improve scripting and its experience in Scribus]] by adding an extension manager and further integration into the GUI and the core. Some examples should show what is possible with scripter.<br />
<br />
=== Add Math Support ===<br />
<br />
[[Add Math Support]]<br />
<br />
* [http://bugs.scribus.net/view.php?id=1030 #1030 - (RFE) Add support for Math]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=SoC2007_ideas&diff=14905SoC2007 ideas2007-03-21T05:34:32Z<p>Ringerc: /* Document-embedded images */</p>
<hr />
<div>[[Category:Scribus_development]] [[Category:GSOC]]<br />
=Google Summer of Code 2007 projects ideas=<br />
<br />
Here is a list of projects that could be done during Google Summer of Code 2007.<br />
<br />
== Scribus Team Ideas ==<br />
<br />
=== Imposition===<br />
<br />
Write an imposition (booklet printing) plugin for Scribus.<br />
<br />
==== Background Information ====<br />
<br />
* [http://en.wikipedia.org/wiki/Imposition Wikipedia]<br />
* [http://desktoppub.about.com/od/imposition/a/imposition_2.htm About.com]<br />
* [http://desktoppub.about.com/od/imposition/Imposition.htm Preparing for imposition (about.com)]<br />
* [http://www.digitalmediadesigner.com/2001/10_oct/features/alap2dn.htm How it is done in Adobe Indesign]<br />
* [http://www.digitalmediadesigner.com/2001/10_oct/features/alap4.htm How it is done in Quark Express]<br />
* Two examples of how it is done in CorelDraw: [http://www.epiloglaser.com/tl_impositionvar.htm 1], [http://www.corel.com/servlet/Satellite?pagename=Corel3/Section/Display&sid=1047024315119&gid=1047024354238&cid=1047024935656&trkid=cgf0505tip1 2]<br />
<br />
=== Text merge ===<br />
<br />
Write a text merge plugin for Scribus. This would allow scribus to read data records from a table and use it for text content or filenames for image frames. It could also provide a mode to print labels etc.<br />
<br />
:There is a [http://bugs.scribus.net/view.php?id=990 mail merge RFE]. One of Czech student is working on it. It looks like it will turn into common usable text merge from various data sources (CSV, XML, db...) --[[User:Subik|Subik]] 13:05, 7 March 2007 (CET)<br />
<br />
=== End-to-end publishing solution ===<br />
<br />
This is a meta-project that could be distilled into one or more workable GSoC projects.<br />
<br />
[[Draft of end-to-end publishing solution|End-to-end publishing solution]]<br />
<br />
[[GSoC 2007 Publishing Solution Task Plan]] - This is the result of a discussion in #scribus on 2007-03-16.<br />
<br />
=== LiveCD ===<br />
[[Live DTP CD|Live DTP CD or DVD]] &ndash; this draft/project is 2 years old and should be reviewed first, but there is definitely a need for such a solution.<br />
<br />
It should be doable within the GSoC timeframe by creating a CDD (Custom Debian Distribution).<br />
<br />
=== Text frame usability enhancement ===<br />
<br />
Improve on-page text editing usability and performance (ringerc).<br />
<br />
=== Enhance inline objects ===<br />
<br />
Enhance inline objects (ringerc).<br />
<br />
Inline objects are supported in Scribus, but not easily discovered. Support for positioning and sizing them is poor, and they aren't editable once they've been pasted into a run of text. Considerable improvement is needed in formatting and editing them to make them more useful.<br />
<br />
=== Merge Field Support ===<br />
<br />
Implement true "merge" style field support / variable text /conditional text a-la FrameMaker (ringerc).<br />
<br />
In technical writing, automated mailing, and certain other uses it is desirable to dynamically alter sections of the text.<br />
<br />
The most common need is to substitute certain values into `fields' in the text, as in a mail merge. Supporting such fields would be very useful, especially as Scribus's support for external data sources improves and it becomes more suitable for automated use.<br />
<br />
The second major need is to handle regions of text that can be hidden or shown based on a variable. This can be useful with technical writing. Consider: maintaining manuals for with-wifi and without-wifi variants of a DSL router in a single document by turning off wifi-related sections when exporting without-wifi version. It's also useful for mail merge, as sometimes you want to include or exclude sections of a merge document based on parameters of the record's information. For example, users paying with different methods might need different information, or maybe you want to tell "premium" customers something else. Whatever.<br />
<br />
Adobe's FrameMaker has supported these features for a long time, but few other tools handle anything like it - let alone tools like Scribus with decent layout capabilities. Many people who need this sort of thing currently use PDF generation libraries to produce their documents using languages like Java.<br />
<br />
=== Document-embedded images ===<br />
<br />
Support loading images from alternate sources. You should handle existing filesystem links and docuement-embedded images ( XML CDATA/PCDATA sections ), and leave room for things like a URL handler for HTTP images.<br />
<br />
=== Externally linked text ===<br />
<br />
Building on the concept of an XML editor (see below), implement support for using text in externally linked files (as we currently do for images). Linking should support normal file system URLs, but leave room for handling WebDAV etc in the future. There is a clear role for this in working with content management systems such as in an end-to-end publishing system.<br />
<br />
== Tales from the Crypt, a.k.a. Ideas from the Bug Tracker ==<br />
<br />
=== XML Editor ===<br />
*[http://bugs.scribus.net/view.php?id=1394 #1394 XML Editor]<br />
* Related to [http://bugs.scribus.net/view.php?id=5190 #5190]<br />
* Related to [http://bugs.scribus.net/view.php?id=2108 #2108]<br />
* Related to [http://bugs.scribus.net/view.php?id=1791 #1791]<br />
<br />
It could be practical to have an on/off toggle in the text editor between Wysiwyg and some kind of XML. One click you have the classical way to type text, Another one, formating disappear, XML tags appear like this :<br />
<br />
<nowiki><br />
--------------------------------------<br />
<font name="Bistream Vera Sans MS" size="10"><br />
blah blah blahalahb blah blah<br />
</font><br />
<justify>blih blih blih blah blah</justify><br />
<style name="paragraph"><br />
ploplpoplpoplploplpo<br />
frudubulubruih :)<br />
</style><br />
--------------------------------------<br />
</nowiki><br />
<br />
notice the difference between the (ugly) formating tags at the beginning, and the descriptive style tags, wich then define the look of the text.<br />
<br />
The ability to expose the internal XML for text objects might also be rather interesting for the Python interface. (ringerc)<br />
<br />
The approaches of Inkscape and InDesign are worth looking at. (C schaefer)<br />
<br />
Applicants could choose between two different approaches here:<br />
<br />
# Write an enhancement to the Story Editor to enable users to edit the XML content of '''text frames'''.<br />
# Write an XML editor to edit the whole Scribus document (as Inkscape does with SVG)<br />
<br />
<br />
<br />
Inkscape's re-architecturing effort: [http://wiki.inkscape.org/wiki/index.php/SubsystemRearchitecture Subsystem Architecture]<br />
<br />
=== OpenOffice import/export extension for the Story Editor ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=5190 Text export feature: ODT or other]<br />
<br />
Start with the OO text import plugin and work out a plugin that would export valid Open Document Format (ODF) xml for re-import into OpenOffice.<br />
<br />
Scribus-specific elements and attributes would need to be stripped off, of course.<br />
<br />
=== Multiple Search and Replace ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1105 #1105 Search and Replace for the whole document]<br />
<br />
Allow users to do multiple search/replace action at once in a document. For each string we’d be able to access all settings available right now for just one text string. This will add computing power to Scribus, from the UI.<br />
<br />
=== Individual snappable baseline grid lines ===<br />
*[http://bugs.scribus.net/view.php?id=2899 #2899 Snap items to baseline grid]<br />
<br />
[[Baseline Grid Enhancements]] - Discussion page.<br />
<br />
=== Snap guides to locked objects ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=3343 Snap guides to items]<br />
<br />
Instead of letting objects snap to guides, sometimes the reverse is needed, namely letting guides snap to (locked) objects. It would be nice to have this, e.g. by dragging guides while pressing the shift key. The reverse mode should be visible in the cursor (other colour or something similar).<br />
<br />
=== Latex Text Frames ===<br />
*[http://bugs.scribus.net/view.php?id=128 #128 TeX rendering engine]<br />
<br />
I think using LaTeX for setting text (including math, footnotes, bibliography and indices) and Scribus for the layout would make a very good match. But that means you want Scribus to control the position, size and shape of the textframes, and having access to Scribus's colors and fonts wouldn't be bad, either. <br />
<br />
I think it would be better to make it easy to use the same fonts for LaTeX frames and non-LaTeX frames. AFAIK font installation was/is a bigger issue in LaTeX than it ever were in Scribus. If there is a way to automate it, we should take it.<br />
<br />
What I thought of was that Scribus should only provide an interface to LaTeX in the first step: Write LaTeX commands in the SE and let LaTeX do the rendering. At this stage, Scribus style options etc. aren't involved at all.<br />
<br />
Still, LaTeX needs to know the pagesize; or you have to use preview.sty to get pages cropped to the bounding box.<br />
<br />
<br />
Hm, I guess you could do similar thinks with my solution. You'd just have a separate set of text options for LaTeX. If you take a look at fntguide.dvi from the LaTeX docs you'll understand why. There might be some Scribus options which could be translated to LaTeX parameters, but I doubt there will ever be a 1:1 correspondence. I'd be happy if I manage to find a way to translate Scribus' fonts into appropiate LaTeX font load commands and find a suitable math font for each...<br />
<br />
=== Auto-change quotes based on language settings ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=2021 #2021 - Auto-change quotes to typographical quotes depending on language settings]<br />
<br />
I'm looking for a way to automatically change quotes (", ') to typographical quotes (unicode 201c/201e etc.). Would be great if I could use the text filters to do something like: "replace every other occurence of " with unicode 201e, the rest with 201c"<br />
<br />
=== Add spellchecking to Story Editor ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=11 Add Spellcheck functionality to Story Editor]<br />
<br />
Using one of the standard libraries existing on Linux (aspell, ispell, MySpell), or possibly even use Enchant [http://www.abiword.org/enchant/]<br />
<br />
=== Automatic Widow/Orphan Control ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=132 132 - Widow/orphan control]<br />
<br />
=== Add a Clone tool as in Inkscape ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1704 Implement a 'clone' tool a la Inkscape]<br />
<br />
If this matched the featureset of Inkscape's clone tools, this would match the features requested in bugs 922*, 923 and 924, and as an extra comment in bug 344.<br />
<br />
=== PDF Export enhancement for PDF embedding ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1699 Enhance PDF exporter to enabled embedding PDF within PDF for PDF 1.4+]<br />
<br />
Enhance PDF exporter to enabled embedding PDF within PDF for PDF 1.4+ per roadmap<br />
<br />
=== Implement JDF support ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=2460 Implement JDF (Job Definition Format)]<br />
<br />
[http://www.cip4.org/documents/jdf_specifications/JDF1.3.pdf JDF] seems to be the coming standard for job tickets. It may be patent encumbered in the US and countries with so-called FTAs with the US, but scribus should definitely offer the possibility to use the format somehow to be considered a "serious" application.<br />
<br />
In particular, Quark has added support for using JDF as a source of preflight information. It's something I've wondered about for a while, and ended up talking to some of the MS Publisher folks about too. It turns out they're not interested in in-app preflight (despite how much easer it'd make life for publishing shops that have to take .pub jobs from users who can't read a job spec to save their lives).<br />
<br />
If implementation of JDF is too large a task to implement during SoC, applicants could consider implementing one of the subsets, like the [http://www.heidelberg.com/www/html/en/content/articles/prinect/ppf Print Production Format (PPF)] or the [http://www.adobe.com/supportservice/custsupport/NOTES/86e2.htm Portable Job Ticket Format (PJTF)].<br />
<br />
=== Rich Text Format import plugin ===<br />
<br />
Implement an RTF importer based on the existing plugins for other formats. (Investigate [http://sourceforge.net/projects/librtf librtf]'s features.)<br />
<br />
=== WordPerfect Format import plugin ===<br />
<br />
Implement a WordPerfect importer based on the existing plugins for other formats. (Investigate [http://sourceforge.net/projects/libwpd/ libwpd]'s features.)<br />
<br />
===Other Import Filters===<br />
<br />
Applicants could also decide to write import plugins for other file formats. Please have look at the [http://wiki.scribus.net/index.php/File_formats_that_should_be_supported_by_Scribus_%28wish_list%29 list of desired import filters] and the [http://wiki.scribus.net/index.php/New_Colours_Format list of colour formats].<br />
<br />
== Blue Sky Ideas ==<br />
<br />
Present the title of your idea here and link to the wiki page where you elaborate on it.<br />
<br />
=== twexter ===<br />
<br />
[http://twext.com/faq.html twext] texts provide [http://sk.com.br/sk-krash.html comprehensible input] for language learners, and can integrate will many softwares (including [http://meta.wikimedia.org/wiki/Summer_of_Code_2007/proposals MediaWiki] and/or the [http://wiki.laptop.org/go/WiXi wiki] for the $100 laptop ).. why not a twexter plugin for Scribus?<br />
<br />
=== Improve scripting ===<br />
<br />
[[Improve scripting and its experience in Scribus]] by adding an extension manager and further integration into the GUI and the core. Some examples should show what is possible with scripter.<br />
<br />
=== Add Math Support ===<br />
<br />
[[Add Math Support]]<br />
<br />
* [http://bugs.scribus.net/view.php?id=1030 #1030 - (RFE) Add support for Math]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=SoC2007_ideas&diff=14904SoC2007 ideas2007-03-21T05:33:46Z<p>Ringerc: /* Merge Field Support */</p>
<hr />
<div>[[Category:Scribus_development]] [[Category:GSOC]]<br />
=Google Summer of Code 2007 projects ideas=<br />
<br />
Here is a list of projects that could be done during Google Summer of Code 2007.<br />
<br />
== Scribus Team Ideas ==<br />
<br />
=== Imposition===<br />
<br />
Write an imposition (booklet printing) plugin for Scribus.<br />
<br />
==== Background Information ====<br />
<br />
* [http://en.wikipedia.org/wiki/Imposition Wikipedia]<br />
* [http://desktoppub.about.com/od/imposition/a/imposition_2.htm About.com]<br />
* [http://desktoppub.about.com/od/imposition/Imposition.htm Preparing for imposition (about.com)]<br />
* [http://www.digitalmediadesigner.com/2001/10_oct/features/alap2dn.htm How it is done in Adobe Indesign]<br />
* [http://www.digitalmediadesigner.com/2001/10_oct/features/alap4.htm How it is done in Quark Express]<br />
* Two examples of how it is done in CorelDraw: [http://www.epiloglaser.com/tl_impositionvar.htm 1], [http://www.corel.com/servlet/Satellite?pagename=Corel3/Section/Display&sid=1047024315119&gid=1047024354238&cid=1047024935656&trkid=cgf0505tip1 2]<br />
<br />
=== Text merge ===<br />
<br />
Write a text merge plugin for Scribus. This would allow scribus to read data records from a table and use it for text content or filenames for image frames. It could also provide a mode to print labels etc.<br />
<br />
:There is a [http://bugs.scribus.net/view.php?id=990 mail merge RFE]. One of Czech student is working on it. It looks like it will turn into common usable text merge from various data sources (CSV, XML, db...) --[[User:Subik|Subik]] 13:05, 7 March 2007 (CET)<br />
<br />
=== End-to-end publishing solution ===<br />
<br />
This is a meta-project that could be distilled into one or more workable GSoC projects.<br />
<br />
[[Draft of end-to-end publishing solution|End-to-end publishing solution]]<br />
<br />
[[GSoC 2007 Publishing Solution Task Plan]] - This is the result of a discussion in #scribus on 2007-03-16.<br />
<br />
=== LiveCD ===<br />
[[Live DTP CD|Live DTP CD or DVD]] &ndash; this draft/project is 2 years old and should be reviewed first, but there is definitely a need for such a solution.<br />
<br />
It should be doable within the GSoC timeframe by creating a CDD (Custom Debian Distribution).<br />
<br />
=== Text frame usability enhancement ===<br />
<br />
Improve on-page text editing usability and performance (ringerc).<br />
<br />
=== Enhance inline objects ===<br />
<br />
Enhance inline objects (ringerc).<br />
<br />
Inline objects are supported in Scribus, but not easily discovered. Support for positioning and sizing them is poor, and they aren't editable once they've been pasted into a run of text. Considerable improvement is needed in formatting and editing them to make them more useful.<br />
<br />
=== Merge Field Support ===<br />
<br />
Implement true "merge" style field support / variable text /conditional text a-la FrameMaker (ringerc).<br />
<br />
In technical writing, automated mailing, and certain other uses it is desirable to dynamically alter sections of the text.<br />
<br />
The most common need is to substitute certain values into `fields' in the text, as in a mail merge. Supporting such fields would be very useful, especially as Scribus's support for external data sources improves and it becomes more suitable for automated use.<br />
<br />
The second major need is to handle regions of text that can be hidden or shown based on a variable. This can be useful with technical writing. Consider: maintaining manuals for with-wifi and without-wifi variants of a DSL router in a single document by turning off wifi-related sections when exporting without-wifi version. It's also useful for mail merge, as sometimes you want to include or exclude sections of a merge document based on parameters of the record's information. For example, users paying with different methods might need different information, or maybe you want to tell "premium" customers something else. Whatever.<br />
<br />
Adobe's FrameMaker has supported these features for a long time, but few other tools handle anything like it - let alone tools like Scribus with decent layout capabilities. Many people who need this sort of thing currently use PDF generation libraries to produce their documents using languages like Java.<br />
<br />
=== Document-embedded images ===<br />
<br />
Support loading images from alternate sources. You should handle existing filesystem links and docuemtn-embedded images ( XML CDATA/PCDATA sections ), and leave room for things like a URL handler for HTTP images.<br />
<br />
=== Externally linked text ===<br />
<br />
Building on the concept of an XML editor (see below), implement support for using text in externally linked files (as we currently do for images). Linking should support normal file system URLs, but leave room for handling WebDAV etc in the future. There is a clear role for this in working with content management systems such as in an end-to-end publishing system.<br />
<br />
== Tales from the Crypt, a.k.a. Ideas from the Bug Tracker ==<br />
<br />
=== XML Editor ===<br />
*[http://bugs.scribus.net/view.php?id=1394 #1394 XML Editor]<br />
* Related to [http://bugs.scribus.net/view.php?id=5190 #5190]<br />
* Related to [http://bugs.scribus.net/view.php?id=2108 #2108]<br />
* Related to [http://bugs.scribus.net/view.php?id=1791 #1791]<br />
<br />
It could be practical to have an on/off toggle in the text editor between Wysiwyg and some kind of XML. One click you have the classical way to type text, Another one, formating disappear, XML tags appear like this :<br />
<br />
<nowiki><br />
--------------------------------------<br />
<font name="Bistream Vera Sans MS" size="10"><br />
blah blah blahalahb blah blah<br />
</font><br />
<justify>blih blih blih blah blah</justify><br />
<style name="paragraph"><br />
ploplpoplpoplploplpo<br />
frudubulubruih :)<br />
</style><br />
--------------------------------------<br />
</nowiki><br />
<br />
notice the difference between the (ugly) formating tags at the beginning, and the descriptive style tags, wich then define the look of the text.<br />
<br />
The ability to expose the internal XML for text objects might also be rather interesting for the Python interface. (ringerc)<br />
<br />
The approaches of Inkscape and InDesign are worth looking at. (C schaefer)<br />
<br />
Applicants could choose between two different approaches here:<br />
<br />
# Write an enhancement to the Story Editor to enable users to edit the XML content of '''text frames'''.<br />
# Write an XML editor to edit the whole Scribus document (as Inkscape does with SVG)<br />
<br />
<br />
<br />
Inkscape's re-architecturing effort: [http://wiki.inkscape.org/wiki/index.php/SubsystemRearchitecture Subsystem Architecture]<br />
<br />
=== OpenOffice import/export extension for the Story Editor ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=5190 Text export feature: ODT or other]<br />
<br />
Start with the OO text import plugin and work out a plugin that would export valid Open Document Format (ODF) xml for re-import into OpenOffice.<br />
<br />
Scribus-specific elements and attributes would need to be stripped off, of course.<br />
<br />
=== Multiple Search and Replace ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1105 #1105 Search and Replace for the whole document]<br />
<br />
Allow users to do multiple search/replace action at once in a document. For each string we’d be able to access all settings available right now for just one text string. This will add computing power to Scribus, from the UI.<br />
<br />
=== Individual snappable baseline grid lines ===<br />
*[http://bugs.scribus.net/view.php?id=2899 #2899 Snap items to baseline grid]<br />
<br />
[[Baseline Grid Enhancements]] - Discussion page.<br />
<br />
=== Snap guides to locked objects ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=3343 Snap guides to items]<br />
<br />
Instead of letting objects snap to guides, sometimes the reverse is needed, namely letting guides snap to (locked) objects. It would be nice to have this, e.g. by dragging guides while pressing the shift key. The reverse mode should be visible in the cursor (other colour or something similar).<br />
<br />
=== Latex Text Frames ===<br />
*[http://bugs.scribus.net/view.php?id=128 #128 TeX rendering engine]<br />
<br />
I think using LaTeX for setting text (including math, footnotes, bibliography and indices) and Scribus for the layout would make a very good match. But that means you want Scribus to control the position, size and shape of the textframes, and having access to Scribus's colors and fonts wouldn't be bad, either. <br />
<br />
I think it would be better to make it easy to use the same fonts for LaTeX frames and non-LaTeX frames. AFAIK font installation was/is a bigger issue in LaTeX than it ever were in Scribus. If there is a way to automate it, we should take it.<br />
<br />
What I thought of was that Scribus should only provide an interface to LaTeX in the first step: Write LaTeX commands in the SE and let LaTeX do the rendering. At this stage, Scribus style options etc. aren't involved at all.<br />
<br />
Still, LaTeX needs to know the pagesize; or you have to use preview.sty to get pages cropped to the bounding box.<br />
<br />
<br />
Hm, I guess you could do similar thinks with my solution. You'd just have a separate set of text options for LaTeX. If you take a look at fntguide.dvi from the LaTeX docs you'll understand why. There might be some Scribus options which could be translated to LaTeX parameters, but I doubt there will ever be a 1:1 correspondence. I'd be happy if I manage to find a way to translate Scribus' fonts into appropiate LaTeX font load commands and find a suitable math font for each...<br />
<br />
=== Auto-change quotes based on language settings ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=2021 #2021 - Auto-change quotes to typographical quotes depending on language settings]<br />
<br />
I'm looking for a way to automatically change quotes (", ') to typographical quotes (unicode 201c/201e etc.). Would be great if I could use the text filters to do something like: "replace every other occurence of " with unicode 201e, the rest with 201c"<br />
<br />
=== Add spellchecking to Story Editor ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=11 Add Spellcheck functionality to Story Editor]<br />
<br />
Using one of the standard libraries existing on Linux (aspell, ispell, MySpell), or possibly even use Enchant [http://www.abiword.org/enchant/]<br />
<br />
=== Automatic Widow/Orphan Control ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=132 132 - Widow/orphan control]<br />
<br />
=== Add a Clone tool as in Inkscape ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1704 Implement a 'clone' tool a la Inkscape]<br />
<br />
If this matched the featureset of Inkscape's clone tools, this would match the features requested in bugs 922*, 923 and 924, and as an extra comment in bug 344.<br />
<br />
=== PDF Export enhancement for PDF embedding ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1699 Enhance PDF exporter to enabled embedding PDF within PDF for PDF 1.4+]<br />
<br />
Enhance PDF exporter to enabled embedding PDF within PDF for PDF 1.4+ per roadmap<br />
<br />
=== Implement JDF support ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=2460 Implement JDF (Job Definition Format)]<br />
<br />
[http://www.cip4.org/documents/jdf_specifications/JDF1.3.pdf JDF] seems to be the coming standard for job tickets. It may be patent encumbered in the US and countries with so-called FTAs with the US, but scribus should definitely offer the possibility to use the format somehow to be considered a "serious" application.<br />
<br />
In particular, Quark has added support for using JDF as a source of preflight information. It's something I've wondered about for a while, and ended up talking to some of the MS Publisher folks about too. It turns out they're not interested in in-app preflight (despite how much easer it'd make life for publishing shops that have to take .pub jobs from users who can't read a job spec to save their lives).<br />
<br />
If implementation of JDF is too large a task to implement during SoC, applicants could consider implementing one of the subsets, like the [http://www.heidelberg.com/www/html/en/content/articles/prinect/ppf Print Production Format (PPF)] or the [http://www.adobe.com/supportservice/custsupport/NOTES/86e2.htm Portable Job Ticket Format (PJTF)].<br />
<br />
=== Rich Text Format import plugin ===<br />
<br />
Implement an RTF importer based on the existing plugins for other formats. (Investigate [http://sourceforge.net/projects/librtf librtf]'s features.)<br />
<br />
=== WordPerfect Format import plugin ===<br />
<br />
Implement a WordPerfect importer based on the existing plugins for other formats. (Investigate [http://sourceforge.net/projects/libwpd/ libwpd]'s features.)<br />
<br />
===Other Import Filters===<br />
<br />
Applicants could also decide to write import plugins for other file formats. Please have look at the [http://wiki.scribus.net/index.php/File_formats_that_should_be_supported_by_Scribus_%28wish_list%29 list of desired import filters] and the [http://wiki.scribus.net/index.php/New_Colours_Format list of colour formats].<br />
<br />
== Blue Sky Ideas ==<br />
<br />
Present the title of your idea here and link to the wiki page where you elaborate on it.<br />
<br />
=== twexter ===<br />
<br />
[http://twext.com/faq.html twext] texts provide [http://sk.com.br/sk-krash.html comprehensible input] for language learners, and can integrate will many softwares (including [http://meta.wikimedia.org/wiki/Summer_of_Code_2007/proposals MediaWiki] and/or the [http://wiki.laptop.org/go/WiXi wiki] for the $100 laptop ).. why not a twexter plugin for Scribus?<br />
<br />
=== Improve scripting ===<br />
<br />
[[Improve scripting and its experience in Scribus]] by adding an extension manager and further integration into the GUI and the core. Some examples should show what is possible with scripter.<br />
<br />
=== Add Math Support ===<br />
<br />
[[Add Math Support]]<br />
<br />
* [http://bugs.scribus.net/view.php?id=1030 #1030 - (RFE) Add support for Math]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=SoC2007_ideas&diff=14903SoC2007 ideas2007-03-21T05:27:20Z<p>Ringerc: /* Enhance inline objects */</p>
<hr />
<div>[[Category:Scribus_development]] [[Category:GSOC]]<br />
=Google Summer of Code 2007 projects ideas=<br />
<br />
Here is a list of projects that could be done during Google Summer of Code 2007.<br />
<br />
== Scribus Team Ideas ==<br />
<br />
=== Imposition===<br />
<br />
Write an imposition (booklet printing) plugin for Scribus.<br />
<br />
==== Background Information ====<br />
<br />
* [http://en.wikipedia.org/wiki/Imposition Wikipedia]<br />
* [http://desktoppub.about.com/od/imposition/a/imposition_2.htm About.com]<br />
* [http://desktoppub.about.com/od/imposition/Imposition.htm Preparing for imposition (about.com)]<br />
* [http://www.digitalmediadesigner.com/2001/10_oct/features/alap2dn.htm How it is done in Adobe Indesign]<br />
* [http://www.digitalmediadesigner.com/2001/10_oct/features/alap4.htm How it is done in Quark Express]<br />
* Two examples of how it is done in CorelDraw: [http://www.epiloglaser.com/tl_impositionvar.htm 1], [http://www.corel.com/servlet/Satellite?pagename=Corel3/Section/Display&sid=1047024315119&gid=1047024354238&cid=1047024935656&trkid=cgf0505tip1 2]<br />
<br />
=== Text merge ===<br />
<br />
Write a text merge plugin for Scribus. This would allow scribus to read data records from a table and use it for text content or filenames for image frames. It could also provide a mode to print labels etc.<br />
<br />
:There is a [http://bugs.scribus.net/view.php?id=990 mail merge RFE]. One of Czech student is working on it. It looks like it will turn into common usable text merge from various data sources (CSV, XML, db...) --[[User:Subik|Subik]] 13:05, 7 March 2007 (CET)<br />
<br />
=== End-to-end publishing solution ===<br />
<br />
This is a meta-project that could be distilled into one or more workable GSoC projects.<br />
<br />
[[Draft of end-to-end publishing solution|End-to-end publishing solution]]<br />
<br />
[[GSoC 2007 Publishing Solution Task Plan]] - This is the result of a discussion in #scribus on 2007-03-16.<br />
<br />
=== LiveCD ===<br />
[[Live DTP CD|Live DTP CD or DVD]] &ndash; this draft/project is 2 years old and should be reviewed first, but there is definitely a need for such a solution.<br />
<br />
It should be doable within the GSoC timeframe by creating a CDD (Custom Debian Distribution).<br />
<br />
=== Text frame usability enhancement ===<br />
<br />
Improve on-page text editing usability and performance (ringerc).<br />
<br />
=== Enhance inline objects ===<br />
<br />
Enhance inline objects (ringerc).<br />
<br />
Inline objects are supported in Scribus, but not easily discovered. Support for positioning and sizing them is poor, and they aren't editable once they've been pasted into a run of text. Considerable improvement is needed in formatting and editing them to make them more useful.<br />
<br />
=== Merge Field Support ===<br />
<br />
Implement true "merge" style field support / variable text /conditional text a-la FrameMaker (ringerc).<br />
<br />
=== Document-embedded images ===<br />
<br />
Support loading images from alternate sources. You should handle existing filesystem links and docuemtn-embedded images ( XML CDATA/PCDATA sections ), and leave room for things like a URL handler for HTTP images.<br />
<br />
=== Externally linked text ===<br />
<br />
Building on the concept of an XML editor (see below), implement support for using text in externally linked files (as we currently do for images). Linking should support normal file system URLs, but leave room for handling WebDAV etc in the future. There is a clear role for this in working with content management systems such as in an end-to-end publishing system.<br />
<br />
== Tales from the Crypt, a.k.a. Ideas from the Bug Tracker ==<br />
<br />
=== XML Editor ===<br />
*[http://bugs.scribus.net/view.php?id=1394 #1394 XML Editor]<br />
* Related to [http://bugs.scribus.net/view.php?id=5190 #5190]<br />
* Related to [http://bugs.scribus.net/view.php?id=2108 #2108]<br />
* Related to [http://bugs.scribus.net/view.php?id=1791 #1791]<br />
<br />
It could be practical to have an on/off toggle in the text editor between Wysiwyg and some kind of XML. One click you have the classical way to type text, Another one, formating disappear, XML tags appear like this :<br />
<br />
<nowiki><br />
--------------------------------------<br />
<font name="Bistream Vera Sans MS" size="10"><br />
blah blah blahalahb blah blah<br />
</font><br />
<justify>blih blih blih blah blah</justify><br />
<style name="paragraph"><br />
ploplpoplpoplploplpo<br />
frudubulubruih :)<br />
</style><br />
--------------------------------------<br />
</nowiki><br />
<br />
notice the difference between the (ugly) formating tags at the beginning, and the descriptive style tags, wich then define the look of the text.<br />
<br />
The ability to expose the internal XML for text objects might also be rather interesting for the Python interface. (ringerc)<br />
<br />
The approaches of Inkscape and InDesign are worth looking at. (C schaefer)<br />
<br />
Applicants could choose between two different approaches here:<br />
<br />
# Write an enhancement to the Story Editor to enable users to edit the XML content of '''text frames'''.<br />
# Write an XML editor to edit the whole Scribus document (as Inkscape does with SVG)<br />
<br />
<br />
<br />
Inkscape's re-architecturing effort: [http://wiki.inkscape.org/wiki/index.php/SubsystemRearchitecture Subsystem Architecture]<br />
<br />
=== OpenOffice import/export extension for the Story Editor ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=5190 Text export feature: ODT or other]<br />
<br />
Start with the OO text import plugin and work out a plugin that would export valid Open Document Format (ODF) xml for re-import into OpenOffice.<br />
<br />
Scribus-specific elements and attributes would need to be stripped off, of course.<br />
<br />
=== Multiple Search and Replace ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1105 #1105 Search and Replace for the whole document]<br />
<br />
Allow users to do multiple search/replace action at once in a document. For each string we’d be able to access all settings available right now for just one text string. This will add computing power to Scribus, from the UI.<br />
<br />
=== Individual snappable baseline grid lines ===<br />
*[http://bugs.scribus.net/view.php?id=2899 #2899 Snap items to baseline grid]<br />
<br />
[[Baseline Grid Enhancements]] - Discussion page.<br />
<br />
=== Snap guides to locked objects ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=3343 Snap guides to items]<br />
<br />
Instead of letting objects snap to guides, sometimes the reverse is needed, namely letting guides snap to (locked) objects. It would be nice to have this, e.g. by dragging guides while pressing the shift key. The reverse mode should be visible in the cursor (other colour or something similar).<br />
<br />
=== Latex Text Frames ===<br />
*[http://bugs.scribus.net/view.php?id=128 #128 TeX rendering engine]<br />
<br />
I think using LaTeX for setting text (including math, footnotes, bibliography and indices) and Scribus for the layout would make a very good match. But that means you want Scribus to control the position, size and shape of the textframes, and having access to Scribus's colors and fonts wouldn't be bad, either. <br />
<br />
I think it would be better to make it easy to use the same fonts for LaTeX frames and non-LaTeX frames. AFAIK font installation was/is a bigger issue in LaTeX than it ever were in Scribus. If there is a way to automate it, we should take it.<br />
<br />
What I thought of was that Scribus should only provide an interface to LaTeX in the first step: Write LaTeX commands in the SE and let LaTeX do the rendering. At this stage, Scribus style options etc. aren't involved at all.<br />
<br />
Still, LaTeX needs to know the pagesize; or you have to use preview.sty to get pages cropped to the bounding box.<br />
<br />
<br />
Hm, I guess you could do similar thinks with my solution. You'd just have a separate set of text options for LaTeX. If you take a look at fntguide.dvi from the LaTeX docs you'll understand why. There might be some Scribus options which could be translated to LaTeX parameters, but I doubt there will ever be a 1:1 correspondence. I'd be happy if I manage to find a way to translate Scribus' fonts into appropiate LaTeX font load commands and find a suitable math font for each...<br />
<br />
=== Auto-change quotes based on language settings ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=2021 #2021 - Auto-change quotes to typographical quotes depending on language settings]<br />
<br />
I'm looking for a way to automatically change quotes (", ') to typographical quotes (unicode 201c/201e etc.). Would be great if I could use the text filters to do something like: "replace every other occurence of " with unicode 201e, the rest with 201c"<br />
<br />
=== Add spellchecking to Story Editor ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=11 Add Spellcheck functionality to Story Editor]<br />
<br />
Using one of the standard libraries existing on Linux (aspell, ispell, MySpell), or possibly even use Enchant [http://www.abiword.org/enchant/]<br />
<br />
=== Automatic Widow/Orphan Control ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=132 132 - Widow/orphan control]<br />
<br />
=== Add a Clone tool as in Inkscape ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1704 Implement a 'clone' tool a la Inkscape]<br />
<br />
If this matched the featureset of Inkscape's clone tools, this would match the features requested in bugs 922*, 923 and 924, and as an extra comment in bug 344.<br />
<br />
=== PDF Export enhancement for PDF embedding ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1699 Enhance PDF exporter to enabled embedding PDF within PDF for PDF 1.4+]<br />
<br />
Enhance PDF exporter to enabled embedding PDF within PDF for PDF 1.4+ per roadmap<br />
<br />
=== Implement JDF support ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=2460 Implement JDF (Job Definition Format)]<br />
<br />
[http://www.cip4.org/documents/jdf_specifications/JDF1.3.pdf JDF] seems to be the coming standard for job tickets. It may be patent encumbered in the US and countries with so-called FTAs with the US, but scribus should definitely offer the possibility to use the format somehow to be considered a "serious" application.<br />
<br />
In particular, Quark has added support for using JDF as a source of preflight information. It's something I've wondered about for a while, and ended up talking to some of the MS Publisher folks about too. It turns out they're not interested in in-app preflight (despite how much easer it'd make life for publishing shops that have to take .pub jobs from users who can't read a job spec to save their lives).<br />
<br />
If implementation of JDF is too large a task to implement during SoC, applicants could consider implementing one of the subsets, like the [http://www.heidelberg.com/www/html/en/content/articles/prinect/ppf Print Production Format (PPF)] or the [http://www.adobe.com/supportservice/custsupport/NOTES/86e2.htm Portable Job Ticket Format (PJTF)].<br />
<br />
=== Rich Text Format import plugin ===<br />
<br />
Implement an RTF importer based on the existing plugins for other formats. (Investigate [http://sourceforge.net/projects/librtf librtf]'s features.)<br />
<br />
=== WordPerfect Format import plugin ===<br />
<br />
Implement a WordPerfect importer based on the existing plugins for other formats. (Investigate [http://sourceforge.net/projects/libwpd/ libwpd]'s features.)<br />
<br />
===Other Import Filters===<br />
<br />
Applicants could also decide to write import plugins for other file formats. Please have look at the [http://wiki.scribus.net/index.php/File_formats_that_should_be_supported_by_Scribus_%28wish_list%29 list of desired import filters] and the [http://wiki.scribus.net/index.php/New_Colours_Format list of colour formats].<br />
<br />
== Blue Sky Ideas ==<br />
<br />
Present the title of your idea here and link to the wiki page where you elaborate on it.<br />
<br />
=== twexter ===<br />
<br />
[http://twext.com/faq.html twext] texts provide [http://sk.com.br/sk-krash.html comprehensible input] for language learners, and can integrate will many softwares (including [http://meta.wikimedia.org/wiki/Summer_of_Code_2007/proposals MediaWiki] and/or the [http://wiki.laptop.org/go/WiXi wiki] for the $100 laptop ).. why not a twexter plugin for Scribus?<br />
<br />
=== Improve scripting ===<br />
<br />
[[Improve scripting and its experience in Scribus]] by adding an extension manager and further integration into the GUI and the core. Some examples should show what is possible with scripter.<br />
<br />
=== Add Math Support ===<br />
<br />
[[Add Math Support]]<br />
<br />
* [http://bugs.scribus.net/view.php?id=1030 #1030 - (RFE) Add support for Math]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=SoC2007_ideas&diff=14902SoC2007 ideas2007-03-21T05:26:03Z<p>Ringerc: /* Scribus Team Ideas */</p>
<hr />
<div>[[Category:Scribus_development]] [[Category:GSOC]]<br />
=Google Summer of Code 2007 projects ideas=<br />
<br />
Here is a list of projects that could be done during Google Summer of Code 2007.<br />
<br />
== Scribus Team Ideas ==<br />
<br />
=== Imposition===<br />
<br />
Write an imposition (booklet printing) plugin for Scribus.<br />
<br />
==== Background Information ====<br />
<br />
* [http://en.wikipedia.org/wiki/Imposition Wikipedia]<br />
* [http://desktoppub.about.com/od/imposition/a/imposition_2.htm About.com]<br />
* [http://desktoppub.about.com/od/imposition/Imposition.htm Preparing for imposition (about.com)]<br />
* [http://www.digitalmediadesigner.com/2001/10_oct/features/alap2dn.htm How it is done in Adobe Indesign]<br />
* [http://www.digitalmediadesigner.com/2001/10_oct/features/alap4.htm How it is done in Quark Express]<br />
* Two examples of how it is done in CorelDraw: [http://www.epiloglaser.com/tl_impositionvar.htm 1], [http://www.corel.com/servlet/Satellite?pagename=Corel3/Section/Display&sid=1047024315119&gid=1047024354238&cid=1047024935656&trkid=cgf0505tip1 2]<br />
<br />
=== Text merge ===<br />
<br />
Write a text merge plugin for Scribus. This would allow scribus to read data records from a table and use it for text content or filenames for image frames. It could also provide a mode to print labels etc.<br />
<br />
:There is a [http://bugs.scribus.net/view.php?id=990 mail merge RFE]. One of Czech student is working on it. It looks like it will turn into common usable text merge from various data sources (CSV, XML, db...) --[[User:Subik|Subik]] 13:05, 7 March 2007 (CET)<br />
<br />
=== End-to-end publishing solution ===<br />
<br />
This is a meta-project that could be distilled into one or more workable GSoC projects.<br />
<br />
[[Draft of end-to-end publishing solution|End-to-end publishing solution]]<br />
<br />
[[GSoC 2007 Publishing Solution Task Plan]] - This is the result of a discussion in #scribus on 2007-03-16.<br />
<br />
=== LiveCD ===<br />
[[Live DTP CD|Live DTP CD or DVD]] &ndash; this draft/project is 2 years old and should be reviewed first, but there is definitely a need for such a solution.<br />
<br />
It should be doable within the GSoC timeframe by creating a CDD (Custom Debian Distribution).<br />
<br />
=== Text frame usability enhancement ===<br />
<br />
Improve on-page text editing usability and performance (ringerc).<br />
<br />
=== Enhance inline objects ===<br />
<br />
Enhance inline objects (ringerc).<br />
<br />
=== Merge Field Support ===<br />
<br />
Implement true "merge" style field support / variable text /conditional text a-la FrameMaker (ringerc).<br />
<br />
=== Document-embedded images ===<br />
<br />
Support loading images from alternate sources. You should handle existing filesystem links and docuemtn-embedded images ( XML CDATA/PCDATA sections ), and leave room for things like a URL handler for HTTP images.<br />
<br />
=== Externally linked text ===<br />
<br />
Building on the concept of an XML editor (see below), implement support for using text in externally linked files (as we currently do for images). Linking should support normal file system URLs, but leave room for handling WebDAV etc in the future. There is a clear role for this in working with content management systems such as in an end-to-end publishing system.<br />
<br />
== Tales from the Crypt, a.k.a. Ideas from the Bug Tracker ==<br />
<br />
=== XML Editor ===<br />
*[http://bugs.scribus.net/view.php?id=1394 #1394 XML Editor]<br />
* Related to [http://bugs.scribus.net/view.php?id=5190 #5190]<br />
* Related to [http://bugs.scribus.net/view.php?id=2108 #2108]<br />
* Related to [http://bugs.scribus.net/view.php?id=1791 #1791]<br />
<br />
It could be practical to have an on/off toggle in the text editor between Wysiwyg and some kind of XML. One click you have the classical way to type text, Another one, formating disappear, XML tags appear like this :<br />
<br />
<nowiki><br />
--------------------------------------<br />
<font name="Bistream Vera Sans MS" size="10"><br />
blah blah blahalahb blah blah<br />
</font><br />
<justify>blih blih blih blah blah</justify><br />
<style name="paragraph"><br />
ploplpoplpoplploplpo<br />
frudubulubruih :)<br />
</style><br />
--------------------------------------<br />
</nowiki><br />
<br />
notice the difference between the (ugly) formating tags at the beginning, and the descriptive style tags, wich then define the look of the text.<br />
<br />
The ability to expose the internal XML for text objects might also be rather interesting for the Python interface. (ringerc)<br />
<br />
The approaches of Inkscape and InDesign are worth looking at. (C schaefer)<br />
<br />
Applicants could choose between two different approaches here:<br />
<br />
# Write an enhancement to the Story Editor to enable users to edit the XML content of '''text frames'''.<br />
# Write an XML editor to edit the whole Scribus document (as Inkscape does with SVG)<br />
<br />
<br />
<br />
Inkscape's re-architecturing effort: [http://wiki.inkscape.org/wiki/index.php/SubsystemRearchitecture Subsystem Architecture]<br />
<br />
=== OpenOffice import/export extension for the Story Editor ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=5190 Text export feature: ODT or other]<br />
<br />
Start with the OO text import plugin and work out a plugin that would export valid Open Document Format (ODF) xml for re-import into OpenOffice.<br />
<br />
Scribus-specific elements and attributes would need to be stripped off, of course.<br />
<br />
=== Multiple Search and Replace ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1105 #1105 Search and Replace for the whole document]<br />
<br />
Allow users to do multiple search/replace action at once in a document. For each string we’d be able to access all settings available right now for just one text string. This will add computing power to Scribus, from the UI.<br />
<br />
=== Individual snappable baseline grid lines ===<br />
*[http://bugs.scribus.net/view.php?id=2899 #2899 Snap items to baseline grid]<br />
<br />
[[Baseline Grid Enhancements]] - Discussion page.<br />
<br />
=== Snap guides to locked objects ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=3343 Snap guides to items]<br />
<br />
Instead of letting objects snap to guides, sometimes the reverse is needed, namely letting guides snap to (locked) objects. It would be nice to have this, e.g. by dragging guides while pressing the shift key. The reverse mode should be visible in the cursor (other colour or something similar).<br />
<br />
=== Latex Text Frames ===<br />
*[http://bugs.scribus.net/view.php?id=128 #128 TeX rendering engine]<br />
<br />
I think using LaTeX for setting text (including math, footnotes, bibliography and indices) and Scribus for the layout would make a very good match. But that means you want Scribus to control the position, size and shape of the textframes, and having access to Scribus's colors and fonts wouldn't be bad, either. <br />
<br />
I think it would be better to make it easy to use the same fonts for LaTeX frames and non-LaTeX frames. AFAIK font installation was/is a bigger issue in LaTeX than it ever were in Scribus. If there is a way to automate it, we should take it.<br />
<br />
What I thought of was that Scribus should only provide an interface to LaTeX in the first step: Write LaTeX commands in the SE and let LaTeX do the rendering. At this stage, Scribus style options etc. aren't involved at all.<br />
<br />
Still, LaTeX needs to know the pagesize; or you have to use preview.sty to get pages cropped to the bounding box.<br />
<br />
<br />
Hm, I guess you could do similar thinks with my solution. You'd just have a separate set of text options for LaTeX. If you take a look at fntguide.dvi from the LaTeX docs you'll understand why. There might be some Scribus options which could be translated to LaTeX parameters, but I doubt there will ever be a 1:1 correspondence. I'd be happy if I manage to find a way to translate Scribus' fonts into appropiate LaTeX font load commands and find a suitable math font for each...<br />
<br />
=== Auto-change quotes based on language settings ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=2021 #2021 - Auto-change quotes to typographical quotes depending on language settings]<br />
<br />
I'm looking for a way to automatically change quotes (", ') to typographical quotes (unicode 201c/201e etc.). Would be great if I could use the text filters to do something like: "replace every other occurence of " with unicode 201e, the rest with 201c"<br />
<br />
=== Add spellchecking to Story Editor ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=11 Add Spellcheck functionality to Story Editor]<br />
<br />
Using one of the standard libraries existing on Linux (aspell, ispell, MySpell), or possibly even use Enchant [http://www.abiword.org/enchant/]<br />
<br />
=== Automatic Widow/Orphan Control ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=132 132 - Widow/orphan control]<br />
<br />
=== Add a Clone tool as in Inkscape ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1704 Implement a 'clone' tool a la Inkscape]<br />
<br />
If this matched the featureset of Inkscape's clone tools, this would match the features requested in bugs 922*, 923 and 924, and as an extra comment in bug 344.<br />
<br />
=== PDF Export enhancement for PDF embedding ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=1699 Enhance PDF exporter to enabled embedding PDF within PDF for PDF 1.4+]<br />
<br />
Enhance PDF exporter to enabled embedding PDF within PDF for PDF 1.4+ per roadmap<br />
<br />
=== Implement JDF support ===<br />
<br />
*[http://bugs.scribus.net/view.php?id=2460 Implement JDF (Job Definition Format)]<br />
<br />
[http://www.cip4.org/documents/jdf_specifications/JDF1.3.pdf JDF] seems to be the coming standard for job tickets. It may be patent encumbered in the US and countries with so-called FTAs with the US, but scribus should definitely offer the possibility to use the format somehow to be considered a "serious" application.<br />
<br />
In particular, Quark has added support for using JDF as a source of preflight information. It's something I've wondered about for a while, and ended up talking to some of the MS Publisher folks about too. It turns out they're not interested in in-app preflight (despite how much easer it'd make life for publishing shops that have to take .pub jobs from users who can't read a job spec to save their lives).<br />
<br />
If implementation of JDF is too large a task to implement during SoC, applicants could consider implementing one of the subsets, like the [http://www.heidelberg.com/www/html/en/content/articles/prinect/ppf Print Production Format (PPF)] or the [http://www.adobe.com/supportservice/custsupport/NOTES/86e2.htm Portable Job Ticket Format (PJTF)].<br />
<br />
=== Rich Text Format import plugin ===<br />
<br />
Implement an RTF importer based on the existing plugins for other formats. (Investigate [http://sourceforge.net/projects/librtf librtf]'s features.)<br />
<br />
=== WordPerfect Format import plugin ===<br />
<br />
Implement a WordPerfect importer based on the existing plugins for other formats. (Investigate [http://sourceforge.net/projects/libwpd/ libwpd]'s features.)<br />
<br />
===Other Import Filters===<br />
<br />
Applicants could also decide to write import plugins for other file formats. Please have look at the [http://wiki.scribus.net/index.php/File_formats_that_should_be_supported_by_Scribus_%28wish_list%29 list of desired import filters] and the [http://wiki.scribus.net/index.php/New_Colours_Format list of colour formats].<br />
<br />
== Blue Sky Ideas ==<br />
<br />
Present the title of your idea here and link to the wiki page where you elaborate on it.<br />
<br />
=== twexter ===<br />
<br />
[http://twext.com/faq.html twext] texts provide [http://sk.com.br/sk-krash.html comprehensible input] for language learners, and can integrate will many softwares (including [http://meta.wikimedia.org/wiki/Summer_of_Code_2007/proposals MediaWiki] and/or the [http://wiki.laptop.org/go/WiXi wiki] for the $100 laptop ).. why not a twexter plugin for Scribus?<br />
<br />
=== Improve scripting ===<br />
<br />
[[Improve scripting and its experience in Scribus]] by adding an extension manager and further integration into the GUI and the core. Some examples should show what is possible with scripter.<br />
<br />
=== Add Math Support ===<br />
<br />
[[Add Math Support]]<br />
<br />
* [http://bugs.scribus.net/view.php?id=1030 #1030 - (RFE) Add support for Math]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=14785Improve scripting and its experience in Scribus2007-03-17T05:07:54Z<p>Ringerc: /* My idea */</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see below]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
[CR: The very first step we need is a sane internal API that we can expose for scripts to use - including SIP wrappers for parts of the GUI that we want to open up for manipulation, etc. Discovery through Qt's parent/child mechanism is limited and clumsy. With a decent C++ API and a SIP wrapper we can start making more sane, powerful scripts - and can probably integrate script management into the existing plugin manager.]<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Improve_scripting_and_its_experience_in_Scribus&diff=14784Improve scripting and its experience in Scribus2007-03-17T05:05:42Z<p>Ringerc: Security issues</p>
<hr />
<div>== Introduction ==<br />
<br />
The potential of the scripting interface seems highly underestimated. You can <br />
* add features without knowing C++ and without recompiling<br />
* make something which directly fits your own needs<br />
* automate repetitive tasks to improve usability and to save time<br />
* test ideas - Python is great for prototyping and sometimes it makes more sense to improve the prototype instead of rewriting everything in C++<br />
<br />
<br />
I actually have written prototypes (among others) of<br />
* new-document-wizards<br />
* spell checkers<br />
* importers for exotic file formats<br />
* connectors to databases and content management systems<br />
* GUI hacks (full screen mode, tear-off menus etc.)<br />
<br />
<br />
== Current problems ==<br />
<br />
As you see in the current state scripter is already powerful but it lacks<br />
* full integration into the GUI and the core: Scripts cannot easily add menu entries or toolbar icons, save settings into the Scribus configuration file, etc. [CR: This will probably need a decently designed public C++ API for Scribus, plus a SIP wrapper for it]<br />
* completeness: While writing scripts you sometimes find things which are still missing. For example there is a function to zoom the document view. But there lacks a function to get the current zoom factor. [CR: Again, need a real public API in C++ to make any kind of completeness effort viable and maintainable]<br />
* ease of use: The script console is really minimalistic and needs improvement. Also there is no standard way to distribute and install scripts. <br />
* security: Scripter should warn you if a script wants to import a module which can access the file system. The user should be asked if he trusts a script. [CR: unfortunately the first part of this is not viable, though the second is - see below]<br />
<br />
== Security ==<br />
<br />
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.<br />
<br />
If you want a secure language for automation, embed Lua, (maybe) Qt Script for Applications (an ECMAScript/JavaScript implementation), or even restricted mode Java. Python is not suitable.<br />
<br />
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.<br />
<br />
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.<br />
<br />
== How to make scripting situation better ==<br />
<br />
'' So what is needed? ''<br />
<br />
=== For the user ===<br />
* a standard way to find and to install extensions and scripts from the web, preferably from a trusted repository <br />
* scripts look and behave as native parts of Scribus<br />
=== For the developer ===<br />
* specifications on how scripts can be integrated with Scribus<br />
* more example scripts / extensions to see what is possible and how to make use of Scribus' features<br />
* completed missing functionally<br />
<br />
<br />
== My idea ==<br />
<br />
'''So I propose writing an extension manager for Scribus together with some example scripts and extensions.'''<br />
<br />
This will probably help to find missing functionally and integration issues. Currently it is very easy to write scripts and this should not become harder by adding this new layer.<br />
<br />
Mozilla Firefox shows what is possible with easily installable extensions. People without knowledge of C-programming can realize crazy ideas and useful features. A lot of the popularity of Firefox comes from its add-ons.<br />
<br />
<br />
== Implementation details ==<br />
<br />
The GUI part will use the PyQt library which can directly integrate with the Scribus GUI. PyQt is available on all platforms where Qt is available and also supports Qt4 which Scribus will soon use. Nearly all Linux distributions contain PyQt and some like Kubuntu install it by default.<br />
<br />
Most parts of the extension manager can be implemented in pure Python and rely on the already available hooks in scripter. Besides some stuff has to be added:<br />
* functions to get the configuration directory and the program directory<br />
* some widgets need to have a name so PyQt can find and access them <br />
* (more to come soon)<br />
<br />
See the links below to see that I already have experience in this area.<br />
<br />
== Links ==<br />
<br />
* [http://henning.cco-ev.de/scribus/screenshots/extensionmanager.jpg Screenshot of an extension manager prototype ]<br />
* [http://henning.cco-ev.de/scribus/screenshots/openclipart.jpg Screenshot of an OpenClipArt importer prototype]<br />
* [http://henning.cco-ev.de/scribus Example for a repository]<br />
* [http://bugs.scribus.net/view.php?id=2160 I once wrote a patch which added the importSVG-function to scripter]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Imposition&diff=6725Imposition2005-12-05T10:38:18Z<p>Ringerc: /* External Tools */</p>
<hr />
<div>=== Imposition ===<br />
<br />
Some thoughts on imposition - approaches, requirements, etc.<br />
<br />
There seem to be two main "classes" of imposition - simple booklet printing [http://bugs.scribus.net/view.php?id=83 #83], and full imposition.<br />
<br />
Booklet printing has simpler requirements. No need for rotation, special page positioning, etc. It's mostly about n-up printing and page order.<br />
<br />
Full imposition can require:<br />
<br />
* page rotation<br />
* custom positioning of pages<br />
* page reordering<br />
* page duplication/repetition<br />
* Addition of cut/crop marks<br />
* Addition of colour bars<br />
* Addition of registration marks<br />
<br />
There are several ways to tackle imposition:<br />
<br />
* With an external tool to post-process PDF or PostScript output (needs external tool to exist and be user friendly, 2 step process)<br />
* As a second stage in Scribus by embedding exported PDF pages into a new PDF (requires PDF-in-PDF embedding, 2 step process)<br />
* As part of PDF export and printing as an extension of the existing page selection/ordering scheme (needs custom "imposition editor" GUI)<br />
* Using the existing canvas and layout tools to position frames representing pages, which when printed pull in those pages. See discussion on [http://bugs.scribus.net/view.php?id=83 #83]. (Requires ability to place a Scribus page - from the same doc, or another doc - into a frame)<br />
<br />
==== External Tools ====<br />
<br />
There's not much we can do directly in this space. A few existing tools exist, such as Multivalent. This is a post-processing step, which offers maximum flexibility but less convenience. There's also the [[How_to_make_impositions_with_pstops|wiki page on using ps2ps]] to make impositions.<br />
<br />
==== PDF-in-PDF embedding / EPS-in-PS embedding ====<br />
<br />
We can already do imposition to PostScript by exporting pages as EPS files. The same sort of thing could be done for PDF once Scribus supports embedding a PDF in an output PDF without rasterisation.<br />
<br />
The downside is that this approach would be clumsy for more than a couple of pages due to the need to manually create the impositions. Support in the page templates for pulling in page "n" of an EPS/PDF, where "n" is a function of the current page number, would help a lot. Scripting would be another way to solve this.<br />
<br />
This approach does require the user to export a document, then create another document for the imposition of it. As a post-processing approach, it's flexible, but somewhat inconvenient. Full support for crop marks, etc, would be possible, as in the documents-as-PageItems case below. As an added bonus, we could impose any PDF/EPS document, not just a Scribus generated one.<br />
<br />
==== Built-in imposition support at PDF/PS export time ====<br />
<br />
Doing imposition during PDF/PS export might be simpler and quicker to implement. On the other hand, for the user to be able to specify the impositions we'd need either a nasty-to-work-with text language to specify page order, position, rotation, etc, or a custom imposition editor GUI. That GUI, while not excessively difficult to write, would either be fairly limited or end up duplicating quite a bit of the functionality of the Scribus canvas.<br />
<br />
This approach would probably be the most convenient and user friendly (ideal for booklet printing etc) but more limited. It would be hard to give the user full flexibility to do crop marks, colour bars, etc to their specs.<br />
<br />
==== Imposition by including Scribus pages in PageItems ====<br />
<br />
This approach would be the most flexible. It could be done either as post-processing step or possibly in the document its self. It'd permit an imposition to include custom elements like registration marks, colour bars, etc set up to the exact specs of the printer by placing them on the page as normal objects. These could be generated by a script, provided in a scrapbook, etc. Additionally, imposition would be very flexible as it'd use the full facilities of the scribus page layout and PageItem transformation. Being able to pull in a document as a frame would be very useful for other things too, including:<br />
<br />
* "Master Document" support<br />
* Pulling in external ads into a page (though this is easily done with EPS, and with PDF-in-PDF embedding when supported, using .sla gives maximum flexiblity at output time)<br />
<br />
Such an imposition would need to be able to be done as some sort of template, where you design the first page and subsequent pages are done the same but with increasing sequence numbers, page numbers of included pages, etc. Some way would be required to specify how the page numbers to include were to increment.<br />
<br />
This approach has a lot in common with using an EPS or PDF in a PageItem, as discussed above.<br />
<br />
This could be done two signficantly different ways:<br />
<br />
* Create a second document for the imposition, and "link" pages from the first document into PageItems like you do an EPS/PDF. Needs the ability for a PageItem to reference/link to a page in an external document.<br />
* Create special "imposition pages" inside the original document, probably working somewhat like page templates. This could get rather complicated when considering automatic generation of the individual pages against a template (as large documents will need) vs hand imposition of each page as might be desired for booklets, etc. Needs the ability for a PageItem to "reference" another page in a document.<br />
<br />
===Additional sources===<br />
* http://www.prepress.pps.com/TechReports/imposetr.html &mdash; See this page if you don't know, what imposition means</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Imposition&diff=5181Imposition2005-12-05T10:38:07Z<p>Ringerc: /* External Tools */</p>
<hr />
<div>=== Imposition ===<br />
<br />
Some thoughts on imposition - approaches, requirements, etc.<br />
<br />
There seem to be two main "classes" of imposition - simple booklet printing [http://bugs.scribus.net/view.php?id=83 #83], and full imposition.<br />
<br />
Booklet printing has simpler requirements. No need for rotation, special page positioning, etc. It's mostly about n-up printing and page order.<br />
<br />
Full imposition can require:<br />
<br />
* page rotation<br />
* custom positioning of pages<br />
* page reordering<br />
* page duplication/repetition<br />
* Addition of cut/crop marks<br />
* Addition of colour bars<br />
* Addition of registration marks<br />
<br />
There are several ways to tackle imposition:<br />
<br />
* With an external tool to post-process PDF or PostScript output (needs external tool to exist and be user friendly, 2 step process)<br />
* As a second stage in Scribus by embedding exported PDF pages into a new PDF (requires PDF-in-PDF embedding, 2 step process)<br />
* As part of PDF export and printing as an extension of the existing page selection/ordering scheme (needs custom "imposition editor" GUI)<br />
* Using the existing canvas and layout tools to position frames representing pages, which when printed pull in those pages. See discussion on [http://bugs.scribus.net/view.php?id=83 #83]. (Requires ability to place a Scribus page - from the same doc, or another doc - into a frame)<br />
<br />
==== External Tools ====<br />
<br />
There's not much we can do directly in this space. A few existing tools exist, such as Multivalent. This is a post-processing step, which offers maximum flexibility but less convenience. There's also the [How_to_make_impositions_with_pstops|wiki page on using ps2ps] to make impositions.<br />
<br />
==== PDF-in-PDF embedding / EPS-in-PS embedding ====<br />
<br />
We can already do imposition to PostScript by exporting pages as EPS files. The same sort of thing could be done for PDF once Scribus supports embedding a PDF in an output PDF without rasterisation.<br />
<br />
The downside is that this approach would be clumsy for more than a couple of pages due to the need to manually create the impositions. Support in the page templates for pulling in page "n" of an EPS/PDF, where "n" is a function of the current page number, would help a lot. Scripting would be another way to solve this.<br />
<br />
This approach does require the user to export a document, then create another document for the imposition of it. As a post-processing approach, it's flexible, but somewhat inconvenient. Full support for crop marks, etc, would be possible, as in the documents-as-PageItems case below. As an added bonus, we could impose any PDF/EPS document, not just a Scribus generated one.<br />
<br />
==== Built-in imposition support at PDF/PS export time ====<br />
<br />
Doing imposition during PDF/PS export might be simpler and quicker to implement. On the other hand, for the user to be able to specify the impositions we'd need either a nasty-to-work-with text language to specify page order, position, rotation, etc, or a custom imposition editor GUI. That GUI, while not excessively difficult to write, would either be fairly limited or end up duplicating quite a bit of the functionality of the Scribus canvas.<br />
<br />
This approach would probably be the most convenient and user friendly (ideal for booklet printing etc) but more limited. It would be hard to give the user full flexibility to do crop marks, colour bars, etc to their specs.<br />
<br />
==== Imposition by including Scribus pages in PageItems ====<br />
<br />
This approach would be the most flexible. It could be done either as post-processing step or possibly in the document its self. It'd permit an imposition to include custom elements like registration marks, colour bars, etc set up to the exact specs of the printer by placing them on the page as normal objects. These could be generated by a script, provided in a scrapbook, etc. Additionally, imposition would be very flexible as it'd use the full facilities of the scribus page layout and PageItem transformation. Being able to pull in a document as a frame would be very useful for other things too, including:<br />
<br />
* "Master Document" support<br />
* Pulling in external ads into a page (though this is easily done with EPS, and with PDF-in-PDF embedding when supported, using .sla gives maximum flexiblity at output time)<br />
<br />
Such an imposition would need to be able to be done as some sort of template, where you design the first page and subsequent pages are done the same but with increasing sequence numbers, page numbers of included pages, etc. Some way would be required to specify how the page numbers to include were to increment.<br />
<br />
This approach has a lot in common with using an EPS or PDF in a PageItem, as discussed above.<br />
<br />
This could be done two signficantly different ways:<br />
<br />
* Create a second document for the imposition, and "link" pages from the first document into PageItems like you do an EPS/PDF. Needs the ability for a PageItem to reference/link to a page in an external document.<br />
* Create special "imposition pages" inside the original document, probably working somewhat like page templates. This could get rather complicated when considering automatic generation of the individual pages against a template (as large documents will need) vs hand imposition of each page as might be desired for booklets, etc. Needs the ability for a PageItem to "reference" another page in a document.<br />
<br />
===Additional sources===<br />
* http://www.prepress.pps.com/TechReports/imposetr.html &mdash; See this page if you don't know, what imposition means</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Imposition&diff=5180Imposition2005-12-05T10:37:44Z<p>Ringerc: /* External Tools */</p>
<hr />
<div>=== Imposition ===<br />
<br />
Some thoughts on imposition - approaches, requirements, etc.<br />
<br />
There seem to be two main "classes" of imposition - simple booklet printing [http://bugs.scribus.net/view.php?id=83 #83], and full imposition.<br />
<br />
Booklet printing has simpler requirements. No need for rotation, special page positioning, etc. It's mostly about n-up printing and page order.<br />
<br />
Full imposition can require:<br />
<br />
* page rotation<br />
* custom positioning of pages<br />
* page reordering<br />
* page duplication/repetition<br />
* Addition of cut/crop marks<br />
* Addition of colour bars<br />
* Addition of registration marks<br />
<br />
There are several ways to tackle imposition:<br />
<br />
* With an external tool to post-process PDF or PostScript output (needs external tool to exist and be user friendly, 2 step process)<br />
* As a second stage in Scribus by embedding exported PDF pages into a new PDF (requires PDF-in-PDF embedding, 2 step process)<br />
* As part of PDF export and printing as an extension of the existing page selection/ordering scheme (needs custom "imposition editor" GUI)<br />
* Using the existing canvas and layout tools to position frames representing pages, which when printed pull in those pages. See discussion on [http://bugs.scribus.net/view.php?id=83 #83]. (Requires ability to place a Scribus page - from the same doc, or another doc - into a frame)<br />
<br />
==== External Tools ====<br />
<br />
There's not much we can do directly in this space. A few existing tools exist, such as Multivalent. This is a post-processing step, which offers maximum flexibility but less convenience. There's also the [[http://wiki.scribus.net/index.php/How_to_make_impositions_with_pstops|wiki page on using ps2ps]] to make impositions.<br />
<br />
==== PDF-in-PDF embedding / EPS-in-PS embedding ====<br />
<br />
We can already do imposition to PostScript by exporting pages as EPS files. The same sort of thing could be done for PDF once Scribus supports embedding a PDF in an output PDF without rasterisation.<br />
<br />
The downside is that this approach would be clumsy for more than a couple of pages due to the need to manually create the impositions. Support in the page templates for pulling in page "n" of an EPS/PDF, where "n" is a function of the current page number, would help a lot. Scripting would be another way to solve this.<br />
<br />
This approach does require the user to export a document, then create another document for the imposition of it. As a post-processing approach, it's flexible, but somewhat inconvenient. Full support for crop marks, etc, would be possible, as in the documents-as-PageItems case below. As an added bonus, we could impose any PDF/EPS document, not just a Scribus generated one.<br />
<br />
==== Built-in imposition support at PDF/PS export time ====<br />
<br />
Doing imposition during PDF/PS export might be simpler and quicker to implement. On the other hand, for the user to be able to specify the impositions we'd need either a nasty-to-work-with text language to specify page order, position, rotation, etc, or a custom imposition editor GUI. That GUI, while not excessively difficult to write, would either be fairly limited or end up duplicating quite a bit of the functionality of the Scribus canvas.<br />
<br />
This approach would probably be the most convenient and user friendly (ideal for booklet printing etc) but more limited. It would be hard to give the user full flexibility to do crop marks, colour bars, etc to their specs.<br />
<br />
==== Imposition by including Scribus pages in PageItems ====<br />
<br />
This approach would be the most flexible. It could be done either as post-processing step or possibly in the document its self. It'd permit an imposition to include custom elements like registration marks, colour bars, etc set up to the exact specs of the printer by placing them on the page as normal objects. These could be generated by a script, provided in a scrapbook, etc. Additionally, imposition would be very flexible as it'd use the full facilities of the scribus page layout and PageItem transformation. Being able to pull in a document as a frame would be very useful for other things too, including:<br />
<br />
* "Master Document" support<br />
* Pulling in external ads into a page (though this is easily done with EPS, and with PDF-in-PDF embedding when supported, using .sla gives maximum flexiblity at output time)<br />
<br />
Such an imposition would need to be able to be done as some sort of template, where you design the first page and subsequent pages are done the same but with increasing sequence numbers, page numbers of included pages, etc. Some way would be required to specify how the page numbers to include were to increment.<br />
<br />
This approach has a lot in common with using an EPS or PDF in a PageItem, as discussed above.<br />
<br />
This could be done two signficantly different ways:<br />
<br />
* Create a second document for the imposition, and "link" pages from the first document into PageItems like you do an EPS/PDF. Needs the ability for a PageItem to reference/link to a page in an external document.<br />
* Create special "imposition pages" inside the original document, probably working somewhat like page templates. This could get rather complicated when considering automatic generation of the individual pages against a template (as large documents will need) vs hand imposition of each page as might be desired for booklets, etc. Needs the ability for a PageItem to "reference" another page in a document.<br />
<br />
===Additional sources===<br />
* http://www.prepress.pps.com/TechReports/imposetr.html &mdash; See this page if you don't know, what imposition means</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Imposition&diff=5177Imposition2005-12-05T10:26:19Z<p>Ringerc: </p>
<hr />
<div>=== Imposition ===<br />
<br />
Some thoughts on imposition - approaches, requirements, etc.<br />
<br />
There seem to be two main "classes" of imposition - simple booklet printing [http://bugs.scribus.net/view.php?id=83 #83], and full imposition.<br />
<br />
Booklet printing has simpler requirements. No need for rotation, special page positioning, etc. It's mostly about n-up printing and page order.<br />
<br />
Full imposition can require:<br />
<br />
* page rotation<br />
* custom positioning of pages<br />
* page reordering<br />
* page duplication/repetition<br />
* Addition of cut/crop marks<br />
* Addition of colour bars<br />
* Addition of registration marks<br />
<br />
There are several ways to tackle imposition:<br />
<br />
* With an external tool to post-process PDF or PostScript output (needs external tool to exist and be user friendly, 2 step process)<br />
* As a second stage in Scribus by embedding exported PDF pages into a new PDF (requires PDF-in-PDF embedding, 2 step process)<br />
* As part of PDF export and printing as an extension of the existing page selection/ordering scheme (needs custom "imposition editor" GUI)<br />
* Using the existing canvas and layout tools to position frames representing pages, which when printed pull in those pages. See discussion on [http://bugs.scribus.net/view.php?id=83 #83]. (Requires ability to place a Scribus page - from the same doc, or another doc - into a frame)<br />
<br />
==== External Tools ====<br />
<br />
There's not much we can do directly in this space. A few existing tools exist, such as Multivalent. This is a post-processing step, which offers maximum flexibility but less convenience.<br />
<br />
==== PDF-in-PDF embedding / EPS-in-PS embedding ====<br />
<br />
We can already do imposition to PostScript by exporting pages as EPS files. The same sort of thing could be done for PDF once Scribus supports embedding a PDF in an output PDF without rasterisation.<br />
<br />
The downside is that this approach would be clumsy for more than a couple of pages due to the need to manually create the impositions. Support in the page templates for pulling in page "n" of an EPS/PDF, where "n" is a function of the current page number, would help a lot. Scripting would be another way to solve this.<br />
<br />
This approach does require the user to export a document, then create another document for the imposition of it. As a post-processing approach, it's flexible, but somewhat inconvenient. Full support for crop marks, etc, would be possible, as in the documents-as-PageItems case below. As an added bonus, we could impose any PDF/EPS document, not just a Scribus generated one.<br />
<br />
==== Built-in imposition support at PDF/PS export time ====<br />
<br />
Doing imposition during PDF/PS export might be simpler and quicker to implement. On the other hand, for the user to be able to specify the impositions we'd need either a nasty-to-work-with text language to specify page order, position, rotation, etc, or a custom imposition editor GUI. That GUI, while not excessively difficult to write, would either be fairly limited or end up duplicating quite a bit of the functionality of the Scribus canvas.<br />
<br />
This approach would probably be the most convenient and user friendly (ideal for booklet printing etc) but more limited. It would be hard to give the user full flexibility to do crop marks, colour bars, etc to their specs.<br />
<br />
==== Imposition by including Scribus pages in PageItems ====<br />
<br />
This approach would be the most flexible. It could be done either as post-processing step or possibly in the document its self. It'd permit an imposition to include custom elements like registration marks, colour bars, etc set up to the exact specs of the printer by placing them on the page as normal objects. These could be generated by a script, provided in a scrapbook, etc. Additionally, imposition would be very flexible as it'd use the full facilities of the scribus page layout and PageItem transformation. Being able to pull in a document as a frame would be very useful for other things too, including:<br />
<br />
* "Master Document" support<br />
* Pulling in external ads into a page (though this is easily done with EPS, and with PDF-in-PDF embedding when supported, using .sla gives maximum flexiblity at output time)<br />
<br />
Such an imposition would need to be able to be done as some sort of template, where you design the first page and subsequent pages are done the same but with increasing sequence numbers, page numbers of included pages, etc. Some way would be required to specify how the page numbers to include were to increment.<br />
<br />
This approach has a lot in common with using an EPS or PDF in a PageItem, as discussed above.<br />
<br />
This could be done two signficantly different ways:<br />
<br />
* Create a second document for the imposition, and "link" pages from the first document into PageItems like you do an EPS/PDF. Needs the ability for a PageItem to reference/link to a page in an external document.<br />
* Create special "imposition pages" inside the original document, probably working somewhat like page templates. This could get rather complicated when considering automatic generation of the individual pages against a template (as large documents will need) vs hand imposition of each page as might be desired for booklets, etc. Needs the ability for a PageItem to "reference" another page in a document.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Very_common_requests&diff=6708Very common requests2005-12-05T09:38:23Z<p>Ringerc: </p>
<hr />
<div>Here are some feature requests that come up extremely frequently:<br />
<br />
* Booklet printing [http://bugs.scribus.net/view.php?id=83 #83] and full [[imposition]]<br />
* Multiple page numbering sequences - [http://bugs.scribus.net/view.php?id=2778 #2778] IN PROGRESS<br />
* Page numbering gaps and different starting points (the "negative page number" hack) - [http://bugs.scribus.net/view.php?id=2778 #2778] IN PROGRESS<br />
* Bleed<br />
* <span style="text-decoration: line-through;">Spot colours [http://bugs.scribus.net/view.php?id=2167 #2167]</span><br />
* Enhanced OpenType feature support (ligatures, pair kerning, etc)<br />
<br />
Please don't add things to this page. You might want to file a feature request on [http:// bugs.scribus.net the bug tracker], after searching to check that nobody else has, or drop in on IRC to see what the folks there think of an idea or suggestion.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Very_common_requests&diff=5001Very common requests2005-10-30T14:56:23Z<p>Ringerc: </p>
<hr />
<div>Here are some feature requests that come up extremely frequently:<br />
<br />
* Booklet printing [http://bugs.scribus.net/view.php?id=83 #83] and full imposition<br />
* Multiple page numbering sequences<br />
* Page numbering gaps and different starting points (the "negative page number" hack)<br />
* Bleed<br />
* <span style="text-decoration: line-through;">Spot colours [http://bugs.scribus.net/view.php?id=2167 #2167]</span><br />
* Enhanced OpenType feature support (ligatures, pair kerning, etc)<br />
<br />
Please don't add things to this page. You might want to file a feature request on [http:// bugs.scribus.net the bug tracker], after searching to check that nobody else has, or drop in on IRC to see what the folks there think of an idea or suggestion.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Very_common_requests&diff=4999Very common requests2005-10-30T08:19:58Z<p>Ringerc: </p>
<hr />
<div>Here are some feature requests that come up extremely frequently:<br />
<br />
* Booklet printing [http://bugs.scribus.net/view.php?id=83 #83] and full imposition<br />
* Multiple page numbering sequences<br />
* Page numbering gaps and different starting points (the "negative page number" hack)<br />
* Bleed<br />
* <span style="text-decoration: line-through;">Spot colours [http://bugs.scribus.net/view.php?id=2167 #2167]</span><br />
<br />
Please don't add things to this page. You might want to file a feature request on [http:// bugs.scribus.net the bug tracker], after searching to check that nobody else has, or drop in on IRC to see what the folks there think of an idea or suggestion.</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=1.3.x_Roadmap&diff=50001.3.x Roadmap2005-10-30T07:49:37Z<p>Ringerc: Added link to spot colour bug</p>
<hr />
<div>=== The Scribus 1.3.x Roadmap ===<br />
<br />
This document forms the basis of our work on Scribus 1.3.x/1.4. As you will see, some of the work has been done already. This document will change over time based on the possibilities to achieve these goals in the versions set out below. There is a [[1.3.x Roadmap Extras]] document which accompanies this one that lists the outstanding bugs or feature requests from the 1.1.x/1.2 series that we aim to fix in the 1.3.x development series. Please place your comments and suggestions on the [[Roadmap Suggestions]] page. The best way to ensure your feature requests are seen by the development team is to file a [http://bugs.scribus.net bug] and use <b>severity</b> of "feature". Screenshots or mock-ups are very helpful. Before filing please search to avoid duplicates.<br />
<br />
=== 1.3.0 ===<br />
<br />
* Give access to the Scribus and document preferences at the same time so they are clearly separated. - DONE<br />
* File verifier to test file properties for print readiness - DONE<br />
* Enhancements to canvas - linked frame markers - DONE<br />
* Some performance improvement with large amounts of text present - DONE for 1.3.0<br />
* Code cleanup - DONE for 1.3.0 <br />
* Rename all variables to English ones (partly done) [http://bugs.scribus.net/view.php?id=1688 #1688] - DONE for 1.3.0<br />
* Move all measurement calculations to units.cpp/.h. [http://bugs.scribus.net/view.php?id=1689 #1689] - DONE<br />
* Move all constants such as AppMode and page sizes etc to be defined for multiple use. - DONE for 1.3.0 <br />
* Options for performance (eg, DPI view in image frames, currently 72) low, med, hi, 72, 144, 300 ? [http://bugs.scribus.net/view.php?id=1691 #1691] - DONE<br />
* [http://bugs.scribus.net/bug_view_advanced_page.php?bug_id=0001038 Export text as outlines] in PDF exporter. - DONE <br />
* True facing pages - DONE<br />
* Arrowhead ability on lines - DONE<br />
* Basic undo/redo framework - DONE for 1.3.0 <br />
* [http://bugs.scribus.net/view.php?id=633 633]! Drop caps not reaching to the top, and other issues - DONE<br />
* Undo/Redo rewrite with actions based steps like Gimp has, down to level of per character in text frames, node edits, etc. See [http://bugs.scribus.net/bug_view_advanced_page.php?bug_id=0000023 bug 23] for 1.3.0 <br />
* Start separating GUI code from internal structures and the canvas - DONE for 1.3.0 <br />
* Add Graphical Import: CMYK-TIFF, CMYK-JPG - DONE<br />
* Add Graphical Import: PSD [http://bugs.scribus.net/view.php?id=114 #114] - DONE<br />
* Keyboard Shortcut Manager (loading, import/export of shortcut sets) [http://bugs.scribus.net/view.php?id=1690 #1690] DONE - also see:[[Inkscape_Interoperation Inkscape and Scribus]] and [[Harmonizing_Scribus/Inkscape_Keyboard_Shortcuts]] and see public wiki<br />
* Suppport more page measure units (Centimeter, Cicero) - DONE<br />
* Text on path issues [http://bugs.scribus.net/view.php?id=171 #171] - DONE<br />
* More Text Import slave plugins: OASIS - DONE<br />
* Extra attributes (RDF and custom) on frames and documents (eg Author tag on a particular frame, or source tag on a frame, or 'DBID' on a doc) - DONE<br />
* Implement PDF 1.5 support for layers - DONE<br />
* 1362 admin/CVS directory must be removed on "make dist" - DONE<br />
* Ability to change GUI language on the fly [http://bugs.scribus.net/view.php?id=1175 #1175] - DONE<br />
* Write a new align/distribute palette - DONE<br />
* Implement a color eyedropper - [http://bugs.scribus.net/view.php?id=1602 1602] insert a color picker to pick colors from pictures etc - DONE<br />
<br />
* Make Story Editor non modal giving easier access to editing text - DONE<br />
* Initial native release on Mac OSX - DONE<br />
* Option for linespacing to be based on font's line spacing, user setting or baseline - DONE<br />
* Enable access to DFONT font files and TTC font collections [http://bugs.scribus.net/view.php?id=1816 #1816] - DONE<br />
* More control over character offsets to baseline, underline, strikethrough positioning, font vertical scaling - DONE<br />
* Preflight tool to verify file correctness for printing/exporting with profiles for various export/print options. - DONE <br />
* Ability to have different margins on each page - DONE<br />
* [http://bugs.scribus.net/view.php?id=237 237] Enable Toggling low-res / high - res display of picture on the canvas - DONE<br />
* [http://bugs.scribus.net/view.php?id=1033 1033]? TIFF not found in non-US path - DONE<br />
* [http://bugs.scribus.net/view.php?id=651 651] Enhancements to the Help Browser - DONE<br />
* [http://bugs.scribus.net/view.php?id=1534 1534] PATCH: Informative user feedback when PDF/X3 and colour management - DONE<br />
* [http://bugs.scribus.net/view.php?id=1552 1552] Smallest word in Hyphenator Settings falls short by one letter - DONE<br />
* [http://bugs.scribus.net/view.php?id=1558 1558] Item > Distribue/Align, should be ever present and a toolbar - DONE<br />
* [http://bugs.scribus.net/view.php?id=1572 1572] Text flows around frame does not work correct in page templates - DONE<br />
* [http://bugs.scribus.net/view.php?id=1574 1574] Story Editor: Deleting words with CTRL+Backspace/CTRL+DEL - DONE<br />
* [http://bugs.scribus.net/view.php?id=1595 1595] insert page number- Menu Item - DONE<br />
* [http://bugs.scribus.net/view.php?id=1594 1594] inserting a page number, by the documented cntrl + # , does not work - DONE<br />
* [http://bugs.scribus.net/view.php?id=1633 1633]-assigned plugins manager - DONE<br />
* [http://bugs.scribus.net/view.php?id=1681 1681] - setBookmarks in scripter - DONE<br />
* [http://bugs.scribus.net/view.php?id=1730 1730] - Distribute/Align doesn't correctly distribute space - DONE<br />
* [http://bugs.scribus.net/view.php?id=1674 1674] - assigned - DONE<br />
* [http://bugs.scribus.net/view.php?id=1670 1670] multipled tab spaces in story editor are ignored in text-frame - DONE<br />
* [http://bugs.scribus.net/view.php?id=1661 1661] Selecting multiple paras and changing style only affects one para - DONE<br />
* [http://bugs.scribus.net/view.php?id=1677 1677] Change "Kerning" to "Tracking" in UI to more accurately reflect function - DONE<br />
<br />
=== 1.3.1 ===<br />
<br />
* Rewrite the plugin API and implement an about system for plugins - DONE<br />
* Implement a new startup dialog with more options for opening recent and existing documents - DONE<br />
* Some means of handling spot colours. GS 8.5x has loads of improvements there [http://bugs.scribus.net/view.php?id=2167 #2167] - DONE<br />
* [http://bugs.scribus.net/view.php?id=405 405] Drag rulers to measure things - DONE<br />
* Enable knowing the GS version in About [http://bugs.scribus.net/view.php?id=2230 #2230] - DONE<br />
* Add autoresizing of the canvas when a user drags an item to the edge - DONE<br />
* Implement a page relative and absolute measurement mode for the rulers - DONE<br />
* Convert Regular Pages to Master Pages [http://bugs.scribus.net/view.php?id=2523 2523] - DONE<br />
* Enable changing pages to be a different page size and orientation [http://bugs.scribus.net/view.php?id=81 81]- DONE<br />
* Implement page layout options for 1,2,3,4 pages wide for folding documents, - DONE<br />
* [http://bugs.scribus.net/view.php?id=1099 1099] Inline graphics - DONE<br />
* [http://bugs.scribus.net/view.php?id=1358 1358] Ability to apply a template to all odd (or even) pages to a range of pages in Page>Apply template - DONE<br />
* [http://bugs.scribus.net/view.php?id=1223 1223] Make non-printable "characters" visible - DONE<br />
* New font selection capability based on gtfonts and Tsoots gettext code [http://bugs.scribus.net/view.php?id=1692 #1692] - DONE<br />
* [http://bugs.scribus.net/view.php?id=1488 1488] Being able to resize a straight line or change its angle from any end-points - DONE<br />
* "Zoom to 100%" button on view status bar see also [http://bugs.scribus.net/view.php?id=315 #315] - DONE<br />
* Include additional code, patches and fixes for Win32 and Mac OSX compatibility - DONE<br />
* Work on CMYK images and profiles - DONE<br />
* Enable embedded ICC Profiles for .jpg, .psd and .tif images in the PDF export - DONE<br />
* Enable the use of the tiffsep device of Ghostscript 8.51 for Print Preview and enable plate preview - DONE<br />
* Enable conversion of spot colours to process colours at export - DONE<br />
* Implement a calendar wizard script - DONE<br />
* Enable optional building with cairo rendering support instead of libart_lgpl - DONE<br />
* Major internal work in: (DONE)<br />
**Separating the base classes of Scribus (ScribusApp, PageItem, ScribusView, ScribusDoc)<br />
**Separating GUI code from internal<br />
**Cleaning code<br />
**Variable renaming<br />
**Run KDE code testing scripts and clean/fix as appropriate<br />
**Conversion of various code and structures into own classes.<br />
**Implement a preferences manager class and move code there as appropriate<br />
<br />
=== 1.3.2 ===<br />
<br />
* Implement environment based dialog button order - IN PROGRESS<br />
* Migrate to a new build system<br />
* Start to break up pageitem.cpp into multiple classes, partly based on previous test work - IN PROGRESS FROM 1.3.1<br />
* Continue separating GUI code from internal structures and the canvas - IN PROGRESS FROM 1.3.1<br />
* New file open and save dialog class to handle multiple file location options, eg local, NFS, kio, WebDAV. These would be used in import and export plugins, SLA open/save, get/save text functions. They would pass a stream pointer/filename etc to the resulting plugin for the operation. [http://bugs.scribus.net/view.php?id=1693 #1693]<br />
* Code cleanup - IN PROGRESS FROM 1.3.1<br />
* Continue to rename all variables to English ones (partly done) [http://bugs.scribus.net/view.php?id=1688 #1688] - IN PROGRESS FROM 1.3.1<br />
* Convert important structs (StVorL, etc) into proper classes with sensible defaults and set methods that sanity-check input [http://bugs.scribus.net/view.php?id=1703 #1703] IN PROGRESS FROM 1.3.0<br />
* Add margin 'skipping' support for facing pages<br />
* Tweak linked frame markers for size and colour [http://bugs.scribus.net/view.php?id=2702 #2702] - DONE<br />
* Set font picker, style picker, etc to display "<multiple>" rather than incorrect info <br />
* Move Load and Save functions into a "master/slave" plugin (like GetText is) to easily support loading of various formats (new, old, complete oo.o, etc). [http://bugs.scribus.net/view.php?id=1702 #1702]<br />
* RGB profiled export for bubblejets (Note: CMYK for PDF-X/3 only!) [http://bugs.scribus.net/view.php?id=1697 #1697], 1271<br />
* Warning on missing fonts in placed/imported EPS/PDF files [http://bugs.scribus.net/view.php?id=1700 #1700]<br />
* Option to print PostScript errors [http://bugs.scribus.net/view.php?id=1441 #1441]<br />
* Responsiveness during long operations. Separate UI thread, progress bars? [http://bugs.scribus.net/view.php?id=722 #722], 726, 1036<br />
* Release procedure documentation<br />
* Enhancements to Undo/Redo - IN PROGRESS FROM 1.3.1<br />
* Fix the file dialog for [http://bugs.scribus.net/view.php?id=1913 #1913]<br />
* SVG Layer support [http://bugs.scribus.net/view.php?id=1700 #1700]<br />
* New file format and DTD, 100% XML compliant [http://bugs.scribus.net/view.php?id=111 #111] assigned to malex<br />
* New Colour Storage Format with various options eg, index, name, number, rgba, cmyk, spots, etc. See [[New Colours Format]] [http://bugs.scribus.net/view.php?id=379 #379] [http://bugs.scribus.net/view.php?id=716 #716]<br />
* Enhancements to canvas (text frames show column markers, better frame control options) <br />
* Enhance Qt pixmap image support (ie, the pixmap based formats Qt doesnt currently support) IN PROGRESS<br />
* Move vector import to master/slave plugin format (EPS/SVG/etc) <br />
* New Start-up Dialog for new users on launch. [http://bugs.scribus.net/view.php?id=1709 #1709]<br />
* Enhance EPS importer to support raster graphics [http://bugs.scribus.net/view.php?id=1698 #1698]<br />
* Enhancement to Scratch Space [http://bugs.scribus.net/view.php?id=1989 #1989]<br />
* Add EXIF support and use thumbnails in file dialog. IN PROGRESS<br />
=== 1.3.3 ===<br />
<br />
* Non latin support [http://bugs.scribus.net/view.php?id=1547 #1547], [http://bugs.scribus.net/view.php?id=1079 #1079] <br />
* Support for OpenType alternate glyphs [http://bugs.scribus.net/view.php?id=1413 #1413]<br />
* Cursor placement fixes (so cursors know more about tabs, columns, alignment etc)<br />
* Selection improvements: [http://bugs.scribus.net/bug_view_advanced_page.php?bug_id=0000502 Multiselection in items Overview ], [http://bugs.scribus.net/bug_view_advanced_page.php?bug_id=0000505 Object selection with Ctrl], [http://bugs.scribus.net/bug_view_advanced_page.php?bug_id=0000506 Un-prompted object resizing], [http://bugs.scribus.net/bug_view_advanced_page.php?bug_id=0000390 Object creation and resize with aspect/angle constraint].<br />
* performance with large amounts of text present<br />
* make edit-in-frame more usable? I know, use SE - but that's not very useful when making hyphenation corrections etc. <br />
* Ability to tweak leading on styled paragraphs -OR- disable leading field when working on styled pars where it won't work<br />
* Updatable Linked Text<br />
<br />
=== 1.3.4 ===<br />
<br />
* Text refinements [http://bugs.scribus.net/view.php?id=415 #415]<br />
* Typographic enhancements<br />
* Hyphenation improvements [http://bugs.scribus.net/view.php?id=417 #417]<br />
* Bullet support in paragraph styles [http://bugs.scribus.net/view.php?id=102 #102] Partly DONE<br />
* Inheritable paragraph styles [http://bugs.scribus.net/view.php?id=363 #363]<br />
* Related to above: [http://bugs.scribus.net/bug_view_advanced_page.php?bug_id=0000363 limited styles], presumably with a 'next-style' attribute. Think 'first paragraph' styles, etc.<br />
* Finer than 0.1pt control over leading, kerning, type size <br />
* Finer than 1% control over text width adjustment (DONE in 1.3.0 to 0.1%)<br />
* Tracking adjustment (ie inter-word spacing) See [http://bugs.scribus.net/bug_view_advanced_page.php?bug_id=0000418 bug 418]<br />
* Manual tracking (tweaking inter-word spacing)? <br />
* Maximum inter-word spacing control, or "willingness to hyphenate" control <br />
* non-paragraph-terminating newline char (CTL-enter?)<br />
* General improvements to text frames with muliple styles<br />
* Control hyphenation as part of para style? <br />
* Character based styles [http://bugs.scribus.net/view.php?id=002 #002], 1076/1077<br />
* Working PyQT for Scripter (see [[Using QT from the Python interface]]) IN PROGRESS <br />
* Scripter functions enhancements (hangs together with "Scribus Objects via Scripter" IN PROGRESS <br />
* Asynchronous or indepently threaded Python execution (qt will provide a lot of this for free without requiring real threads ... if it can be made to work) (see patches in [[Using QT from the Python interface]]) IN PROGRESS <br />
* Access to the Scribus core from python outside of Scribus GUI/scripter. (DCOP? CORBA/XML-RPC?) [http://bugs.scribus.net/view.php?id=1675 #1675]<br />
* PDF-in-PDF embedding of placed PDFs on PDF export (works only in PDF-1.4 and above FS) [http://bugs.scribus.net/view.php?id=1699 #1699] see also [http://bugs.scribus.net/view.php?id=1179 #1179]<br />
<br />
=== 1.3.5 ===<br />
* Re-write the scrapbook with SAX [http://bugs.scribus.net/view.php?id=2054 #2054] also # 2457<br />
* Transparency flattener [http://bugs.scribus.net/view.php?id=876 #876], 377<br />
* Ability to clone frames like Inkscape can, allowing, for example, text frames on various layers to have the same size and attributes, except different content (eg layers for various translations of a doc). When one is moved, they all move, when one is resized they all resize etc. .."Linked Layer Objects" ? [http://bugs.scribus.net/view.php?id=1704 #1704]<br />
* Boolean Object methods (like Inkscape etc)<br />
* More Text Import slave plugins: XML/HTML/(XHTML+CSS)/OO.org Calc/Kword/Kspread<br />
<br />
=== 1.3.6 ===<br />
<br />
* Crop, fold, registration capability [http://bugs.scribus.net/view.php?id=194 #194],1113,1116 <br />
* Bleed capability [http://bugs.scribus.net/view.php?id=129 #129], 1041<br />
* Transparency enhancements<br />
<br />
=== 1.3.7 ===<br />
<br />
* Tables rewrite/feature enhancement<br />
* New dialog class for standard look and feel<br />
* Configurable multitab notebook for the Scrapbook, with tabs named by the user for various applications or their choice, eg one tab per doc.<br />
* Multiple scrapbooks open at once <br />
<br />
=== 1.3.8 ===<br />
<br />
* Shortcut profiles, eg "for adobe users", "for xpress users" IN PROGRESS<br />
* Ability to create TOC and Indexes, footnotes etc 1371 IN PROGRESS, TOC done.<br />
* LaTeX support of some sort<br />
* Formula import/input via LaTeX or MathML [http://bugs.scribus.net/view.php?id=1030 #1030]<br />
* Mail merge<br />
* Spellchecker based on Enchant, aspell, or ispell (like it's done in KDE?)<br />
<br />
=== 1.3.9 ===<br />
<br />
* OpenClipArt Integration [http://bugs.scribus.net/view.php?id=1263 #1263] IN PROGRESS<br />
* Font Finder. As manually run item, or when a font is missing in a document, we can provide a method for a user to download and install a font from ANY of the free distro font repositories. Requires: List of fonts and repositories (and the code of course)<br />
<br />
=== 1.3.10 ===</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Category:HOWTO&diff=5074Category:HOWTO2005-10-23T06:45:34Z<p>Ringerc: Reverted edit of Vasa16, changed back to last version by Paraplegic racehorse</p>
<hr />
<div>'''Other languages:''' [[:Category:JTZ|Polski (pl)]] [[:Category:Tutoriais|Portuguese (pt_BR)]]<br />
<br />
<br />
== Installation ==<br />
* [[Installing_Scribus_on_Mac_OS_X |Installing Scribus on Mac OS X (Fink/X11)]]<br />
* [[Installing Scribus from CVS sources on Mac OS X |Installing Scribus from CVS sources on Mac OS X (Fink/X11)]]<br />
* [[Installing Scribus from CVS sources on Mac OS X/Aqua |Installing Scribus from CVS sources on Mac OS X (native version / Aqua)]]<br />
* [[Installing Scribus on Win32|Installing Scribus on Win32 (Cygwin/X11)]]<br />
* [[Compiling LPROF-1.09 on Debian Sarge Linux 2.6.8-1-386]]<br />
* [[Scribus on Debian GNU/Linux]]<br />
* [[Using klik with Scribus]] &mdash; testdriving Scribus the easy way<br />
<br />
== Usage ==<br />
<br />
* [[Laying out articles]]<br />
* [[Simple Scribus Spot Colour Printing method]]<br />
* [[Creating a Table of Contents]]<br />
* [[How to create lists in scribus 1.2x]]<br />
* [[Building Tables]]<br />
<br />
== PDF issues ==<br />
<br />
* [[Your first PDF form with Scribus]] - Learn, how to create editable fields and clickable buttons with Scribus<br />
<br />
== DTP ==<br />
* [[Creating a brochure]] <br />
* [[How to make a booklet]]<br />
* [[How to make impositions with pstops]]<br />
* [[Right font 4 right purpose]]<br />
<br />
== Other ==<br />
* [[Converting Mac fonts for use with Scribus]]<br />
* [[typography|Typography – Make your text look good]]<br />
* [[FontSample.py -- Minor amendments]]<br />
* [[Spell checking your Scribus document]]<br />
* [[Import Publisher to Scribus]]</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Installing_Scribus_1.3.4.x_or_lower_from_CVS_sources_on_Mac_OS_X/Aqua&diff=5205Installing Scribus 1.3.4.x or lower from CVS sources on Mac OS X/Aqua2005-10-23T06:45:31Z<p>Ringerc: Reverted edit of Vasa16, changed back to last version by TomK32</p>
<hr />
<div>[[Category:MacOS]]<br />
{{HOWTO Index}}<br />
----<br />
<br />
=Introduction=<br />
Thanks to Andreas Vox and Craig Ringer, recent Scribus compiles straight from CVS on Mac/Aqua. Andreas Vox periodically delivers updated preview binary bundles which can be simply dropped into the system &mdash; please download it from [http://aqua.scribus.net aqua.scribus.net]. If some daring alpha testers use it and [http://bugs.scribus.net report all errors to the Scribus bug submission website], we might have a fully functional Aqua version when 1.3.0 final comes out. :-)<br />
<br />
Scribus 1.3.0cvs is not for production use.<br />
The current build on Mac/Aqua has some problems of its own:<br />
<br />
* It's slow (Part of this may be to do with menus, part is the sub-optimal canvas code)<br />
* Dialogs behave strangely (no close button, menu disappears, ...)<br />
* Focus problems with Edit mode (use toolbar button instead of double-click)<br />
* Aqua buttons need more space than X11 ones, that causes layout problems, notably with the page selector.<br />
* Needs a special freetype build to access native Mac fonts.<br />
* It's not really tested, so there may be much more.<br />
<br />
=Installation=<br />
<br />
==What to do==<br />
<br />
1. Install Qt/Mac (not the X version!)<br />
<br />
2. Install Freetype 2. <br />
We recommend to build it yourself with "./configure --enable-old-mac-fonts"<br />
<br />
3. Install Fontconfig 2<br />
<br />
4. Check that "freetype-config" and "pkgconfig fontconfig" report those libraries.<br />
<br />
5. Install libart, lcms, libtiff, libjpeg, and libpng via fink.<br />
<br />
6. Checkout Scribus 1.3 from cvs or use a snapshot<br />
<br />
7. Set some environment variables:<br />
QTDIR=/absolut/path/to/qt/dir<br />
CPPFLAGS=-I/sw/lib <br />
LDFLAGS=-L/sw/lib <br />
export CPPFLAGS LDFLAGS QTDIR<br />
<br />
8. run "make -f Makefile.cvs"<br />
<br />
9. unpack the Scribus.app skeleton<br />
<br />
10. make sure you have gcc/g++ 3.3 and not gcc/g++ 4.0 which comes with<br />
Tiger by default.<br />
<br />
11. run "./configure --enable-mac --enable-bundle --enable-debug \<br />
--prefix=/absoute/path/to/Scripus.app/Contents/ \<br />
--with-extra-libs=/sw/lib --with-extra-includes=/sw/include"<br />
** --enable-debug is mandatory since ld chokes on '-s' when linking modules<br />
** either CPP/LDFLAGS or --extra-include/libs might be superflous but I didn't try that<br />
<br />
12. make<br />
13. make install<br />
14. open Scribus.app :-)<br />
<br />
You can also do it the old way: <br />
- leave out "--enable-bundle" in step 10 and choose another prefix<br />
- copy scribus/scribus to /absoute/path/to/Scripus.app/Contents/MacOS/scribus<br />
<br />
== other problems that might occur ==<br />
* First of all the compiler, for me g++-4.0 didn't work so I had to symlink /usr/bin/g++ to /usr/bin/g++3<br />
* Qt 3.3.4 can be found as binary package at http://naranja.umh.es/~atg/software.html<br />
* Python might miss the /usr/lib/libpython2.3.a the solution is a symlink to /usr/lib/libpython2.3.dylib<br />
<br />
==Some caveats==<br />
* Some dialogs don't have a close button, ESC usually works though.<br />
* Preferences and About dialogs are now in menu "scribus"!<br />
** This is by design. It's done automatically by Qt to fit MacOS/X interface standards. "NOTABUG".<br />
* <fill in yourself><br />
<br />
'''See also:'''<br />
* [http://bugs.scribus.net/view.php?id=2015 Summary bug report for Scribus/Aqua]<br />
* [[Installing Scribus on Mac OS_X]] - Howto on installing Scribus (Fink/X11)<br />
* [[MacOSX version of Scribus]] - Ongoing discussion on bringing a native version of Scribus to Mac</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Similar_Programs&diff=6642Similar Programs2005-10-23T06:45:28Z<p>Ringerc: Reverted edit of Vasa16, changed back to last version by C schaefer</p>
<hr />
<div>Here is a list for similar programs that exist beneath Scribus.<br />
<br />
See also a related listing of commercial DTP tools at [http://www.scribus.org.uk/modules.php?op=modload&name=Web_Links&file=index web links on www.scribus.net]<br />
<br />
== Pure DTP/Page layout ==<br />
<br />
; Adobe InDesign (Win32, MacOS X) http://www.adobe.com/products/indesign/ <br />
: Major player<br />
<br />
; Adobe PageMaker (Win32, MacOS X) http://www.adobe.com/products/pagemaker/<br />
: Largely deprecated in favour of InDesign.<br />
<br />
; Apple Pages (MacOS X) http://www.apple.com/au/iwork/pages/<br />
: Newly introduced for enduser and part of the iWork package.<br />
<br />
; Deneba Canvas (Win32, MacOS X) http://www.deneba.com/<br />
: Page layout software with a huge amount of additional features for Win32 and MacOS X.<br />
<br />
; invers Calamus (Win32, MacOS 7 - 9, MacOS X, Atari ST) http://www.calamus.net/<br />
: Page layout software originally developed for Atari ST, but still in development and for sale. '''Very''' strange UI.<br />
<br />
; Microsoft Publisher http://www.microsoft.com<br />
: An extension to MS Office. Sold as an easy-to-use DTP software, but generally considered a bad joke in DTP business.<br />
<br />
; QuarkXPress (Win32, MacOS X) http://www.quark.com/<br />
: Major player<br />
<br />
; Passepartout http://www.stacken.kth.se/project/pptout/<br />
: Open Source using GTK+ in a very early stage.<br />
<br />
; RagTime Solo (Win32, MacOS X) http://www.ragtime-online.com/<br />
: So called "Office Publishing" software. A mixture between office suite and and a page layout programme. Free as in beer for personal use.<br />
<br />
; VivaDesigner (Linux, Win32, MacOS X) http://www.vivadesigner.de<br />
: is available for download as a beta version, you need to register and then you will be emailed a valid licence key every 30 days.<br />
<br />
== Long-document and flexible wordprocessing tools ==<br />
; Adobe FrameMaker (Win32, MacOS X) http://www.adobe.com/products/framemaker/<br />
<br />
; Corel Ventura (Win32) http://www.corel.com<br />
<br />
; Papyrus Office (Win32, MacOS X) http://www.rom-logicware.com<br />
<br />
; OpenOffice.org Writer (Linux, Win32, Solaris, MacOS X) http://www.openoffice.org<br />
<br />
<br />
;TeX/LaTeX<br />
* [http://www.latex-project.org/ LaTeX]<br />
* [http://www.miktex.org/ MiKTeX]<br />
<br />
* [http://kile.sourceforge.net Kile]<br />
* [http://www.lyx.org/ LyX]<br />
* [http://www.texmacs.org/ TeXMacs]<br />
* [http://www.xm1math.net/texmaker/ TeXMaker]<br />
<br />
== Flexible (hybrid vector/wp/DTP) ==<br />
<br />
'''Cenon''' [http://www.cenon.info/ www.cenon.info]<br />
Coded with GNUstep suite, this app available for Linux, Windows, MacOS X. General graphics and layout.<br />
<br />
'''deneba Canvas''' ''see above''<br />
<br />
'''Stone Create''' [http://www.stone.com/Create/Create.html www.stone.com]<br />
Originally coded for NeXT and then migrated to MacOS X. Part of the Stone Studio suite. General graphics and layout (print and web)<br />
<br />
== PDF/PS tools ==</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Man_page_draft_for_1.2.x&diff=6407Man page draft for 1.2.x2005-10-23T06:45:28Z<p>Ringerc: Reverted edit of Vasa16, changed back to last version by Mhanski</p>
<hr />
<div>'''Other languages:''' [[Projekt podręcznika systemowego dla Scribusa 1.2.x|Polski (pl)]]<br />
<br />
=Introduction=<br />
This draft is meant to follow closely the current Scribus development, feel free to add your remarks and update it (provided, you know, what you are writing about :). Translators who want to create a man page in their language should download this original [http://wiki.scribus.net/uploads/scribus.1 Scribus man page], translate its contents, and proceed according to this [http://linux.omnipotent.net/article.php?article_id=12493 man page tutorial], before uploading your new man page to [http://bugs.scribus.net bugs.scribus.net].<br />
<br />
=Man page draft=<br />
==Section==<br />
scribus(1) <br />
<br />
==NAME==<br />
scribus - open source X11-based WYSIWYG desktop publishing app<br />
<br />
==SYNOPSIS==<br />
scribus [-h|--help] [-v|--version] [-l|--lang language] [-la|--langs-<br />
available] [filename]<br />
<br />
==DESCRIPTION==<br />
Scribus is an open source desktop publishing program. It provides true<br />
WYSIWYG editing, excellent PDF export facilities, and a wide range of<br />
other input and output options.<br />
<br />
Please see the scribus documentation at http://docs.scribus.net/ or<br />
the program's built in help for more comprehensive and up-to-date doc-<br />
umentation.<br />
<br />
This man page only provides summaries of some aspects of the program's<br />
use. It's main purpose is to make sure you can find the full documen-<br />
tation quickly and easily.<br />
<br />
==OPTIONS==<br />
The definitive options are in the program's usage statement. Run:<br />
scribus --help to view the usage statement.<br />
<br />
-l, --lang xx<br />
Overrides the system locale and runs Scribus in language xx.<br />
The language is specified with the same POSIX language codes<br />
that are used in the LANG and LC_ALL environment variables. For<br />
example, English can be selected with 'en' (generic English),<br />
'en_GB' (British English), 'en_US' (American english), etc.<br />
Similarly, Deutsch can be selected with 'de' or 'de_DE'.<br />
<br />
-la, --langs-available<br />
Print a list of languages for which user interface translations<br />
are available. To use that language run Scribus as 'scribus -l<br />
xx' where xx is the short language code, or set the locale<br />
environment variables as described below.<br />
<br />
-v, --version<br />
Prints the Scribus version number and exits.<br />
<br />
<br />
-h, --help<br />
Print a brief usage summary.<br />
<br />
==ENVIRONMENT==<br />
Scribus respects the standard locale environment variables. Others may<br />
be used through underlying libraries, such as Qt, or by programs used<br />
by Scribus. http://docs.scribus.net/ and the in-program documentation<br />
may note other environment variables used by Scribus or the programs<br />
and libraries it uses.<br />
<br />
LC_ALL, LC_MESSAGES, LANG<br />
POSIX locale. See locale(1). Scribus uses these three environ-<br />
ment variables, in the listed order of precedence, to select<br />
the language (including the user interface translation) to use.<br />
If none are set, it will fall back on the locale set by the<br />
underlying Qt library.<br />
<br />
PATH Scribus may search the PATH for external tools if their paths<br />
are not specified absolutely. This is currently most likely to<br />
affect you if you have multiple copies of gs(1) installed. You<br />
can set an absolute path to the correct gs(1) in your Scribus<br />
preferences and bypass the PATH search.<br />
<br />
A few important environment variables from other programs are sum-<br />
marised here for convenience, though you should check the original<br />
program's documentation if you are having trouble.<br />
<br />
GS_FONTPATH<br />
GhostScript font path. Affects the font search path for<br />
GhostScript, a component Scribus uses for some PostScript oper-<br />
ations. Add new directories containing fonts to this colon-sep-<br />
arated list to help GhostScript find fonts in non-standard<br />
locations. See gs(1) and the GhostScript HTML documentation for<br />
more information.<br />
<br />
GS_LIB GhostScript library path. GhostScript searches this path for<br />
Fontmap files. Like GS_FONTPATH it's a colon separated list of<br />
directories. You will usually want to use GS_FONTPATH instead,<br />
though building a Fontmap file and using GS_LIB can speed<br />
things up if you have a lot of fonts. See gs(1) and the<br />
GhostScript HTML documentation for more information.<br />
<br />
==FILES==<br />
$HOME/.scribus/<br />
User preferences are stored in $HOME/.scribus/ . Most other<br />
paths are configurable from the Scribus preferences.<br />
<br />
$HOME/.scribus/scribus.rc <br />
Scribus preferences file - formatted in a custom xml schema. <br />
May eventually be retired in favour of new format prefs file. <br />
<br />
$HOME/.scribus/prefs.xml <br />
New format prefs file. As of time of writing (v1.2.x) <br />
this file was mostly used for the storage of window locations <br />
and previously used directories.<br />
<br />
==SEE ALSO==<br />
The scribus documentation site http://docs.scribus.net/ and home page<br />
http://www.scribus.net/<br />
<br />
gs(1), the gs html docs, and http://www.ghostscript.com/<br />
<br />
The Qt library - http://www.trolltech.com/<br />
<br />
==RELATED SOFTWARE AND AFFILIATES==<br />
inkscape(1) - http://inkscape.org/<br />
<br />
OpenClipArt - http://openclipart.org/<br />
<br />
Other affiliates may be listed in the footer on<br />
http://www.scribus.net/<br />
<br />
gimp(1) - http://www.gimp.org/<br />
<br />
==BUGS==<br />
See http://bugs.scribus.net/ for the Scribus on-line status tracker,<br />
which is used to track bug reports and feature requests. If you<br />
intend to submit a bug report or feature request, please search the<br />
bug database first.<br />
<br />
==AUTHORS==<br />
Please see the About Scribus option in the Help menu for a list of<br />
authors, translators, and contributors.<br />
<br />
Updated 2005-05-06 scribus(1)<br />
<br />
=MAN PAGE LICENCE=<br />
*Note as an exception to the general CC license the wiki is using, this man file is licensed under GPL V.2, as are the main source licenses.--[[User:Mrdocs|Mrdocs]] 11:45, 21 Apr 2005 (UTC)</div>Ringerchttps://wiki.scribus.net/wiki/index.php?title=Scribus_Wiki:About&diff=6440Scribus Wiki:About2005-10-23T06:45:25Z<p>Ringerc: Reverted edit of Vasa16, changed back to last version by Mhanski</p>
<hr />
<div></div>Ringerc