Scripter Security Discussion
Python has absolutely no way to ensure that code can be "checked" a-la perl restricted mode or Java's bytecode security model. The restricted python mode was proven to be ineffective and has been removed. Every suggestion anybody has ever come up with as to how to restrict Python programs with the CPython interpreter has been proven to be breakable (see innumerable discussions on the newsgroup - I won't repeat the whole large topic here). Under these circumstances we would be instilling a false sense of security by offering any sort of restricted mode to users.
What we need to do is ensure that documents either (a) can NEVER embed scripts, or (b) can embed scripts but ONLY run them if the user specifically turns that on in the prefs AND accepts a prompt when opening the document. We might use a signing scheme to permit the prompt to be disabled for trusted script publishers as nominated by the user. This prevents the document virus problem we see in MS Office.
Personally I don't think we should permit script embedding in documents at all. Make the user install the script and understand that by doing so they give it the same rights as any other program they install on the system. If we want to support script embedding, we're better off choosing a more restricted language for that and leaving Python for extending the app and for advanced scripting with GUIs etc.
Answer to CR
I am aware of the problems with restricted execution and I also believe there is no ideal solution. But I think the security issue needs to be addressed nevertheless and a little more security is better than no security. In most cases insecure scripts are just bad code containing errors which might harm the system and are no trojans. So:
- Warn the user that an unsigned script cannot be trusted and that he runs it on his own risk and that the script can do everything the current user can do. (Most users are not aware that this is possible at all). [CR: Agreed]
- Use import hooks and tell the user that a script e.g. accesses the "os" or "posix" module. [CR: Couldn't disagree more. See the security notes.]
- Wrap builtins like "open". There are ways to circumvent the wrapped functions (see link below) but only trojans do this and it would a least help for simple scripts. [CR: As per last. ]
- Make a repository of trusted scripts (needs some kind of skilled moderators). [CR: This seems like an excellent idea to me, and would fit in well with a signing scheme.]
- Evaluate RestrictedPython from Zope which proves to work in Zope and is not related to the old rpython/Bastion modules.
I very strongly disagree with the idea of trying to use import hooks and builtin wrappers to provide any sort of access control - or even for information purposes. Such techniques are easily bypassed by malicious users, and more importantly will impart a false sense of security to users. Furthermore, if we have an effective restricted execution environment such measures will be unnecessary, since programs *can't* access the file system etc.
RestrictedPython looks more interesting, in that it takes a saner "ban it until it's proven safe" approach. I'm still not overly convinced, though, and will want to see some strong evidence it's solid. Preferably a BFDL endorsement ;-) . In short, I don't have hard evidence against it yet but past experience with people's attempts to create restricted Python runtimes makes me dubious. The last thing we want is anything that makes users think running scripts from Joe Random is safe if we can't absolutely guarantee that's the case.
In particular, I think that combining the idea of restricted execution with any use of SIP or PyQt is raving lunacy... somewhat akin to giving a loaded handgun to a toddler, but saying it's safe because the safety catch is on. PyQt in particular gives the programmer pretty much free reign to do whatever they want.
The Scribus team has long wanted to get rid of the current Python interface completely, and replace it with a scratch rewrite. To do this, we've been planning on writing a pure C++ wrapper around Scribus's internal implementation. This interface will be available for plugins to use directly, and will be friendly for wrapper generators like SIP. That means that a Python interface will have similar capabilities to what plugins can do (especially if we use SIP and PyQt). It'd save a bunch of work, produce a much cleaner and massively more functional Python interface - and it'll do so without all the hideous hand-coded functions we currently have. Users could use Python's database interface modules and all its other functionality to create very powerful extensions to Scribus, including whole new dialogs etc (with PyQt) and extensions to select parts of the current Scribus GUI*. They'd also be able to make use of hooks to listen on certain events in the Scribus core and (maybe) redirect or block them. By using SIP to wrap the C++ interface we can actually do a lot of this quite easily.
The downside of the above is that making it secure is almost certainly wishful thinking.
I think a limited and more secure scripting interface is a completely different problem to providing a more complete and powerful scripting interface for Python. These two functions really couldn't be more different, and I honestly think we'd be better off providing totally different interfaces for them. Perhaps this can be done with Python too (in the _same_ interpreter instance as the unrestricted mode, remember) ... but I'm still not convinced. I remain of the view that such a restricted interface would be much better done using something like lua+SWIG or Qt Script for Applications. Both lua and QSA are built on more of a "you can't do it unless the developer exposes it" model rather than a completely open model like Python.
Another issue you're probably not familiar with is that of sub-interpreters. Python's sub-intepreter feature (used to provide isolation of different scripts) is not really designed for single-threaded use in an app like Scribus. It's more for things like mod_python in servers. Sub interpreters are causing us a lot of pain in the scripter at the moment, and do not look like they'll ever get improved in Python so they'll fit our needs. Things like PyQt simply do not work within subinterpreters. However, if we drop the use of sub interpreters, scripts will need to be very aware of the shared __main__ namespace, shared module state, etc, and will need to be written more carefully. Currently we support both - the user can run a script in the top level interpreter (for PyQt use etc) or in a subinterpreter. This causes its own problems.
I would like to stress at this point that I am EXTREMELY happy that you're interested in the scripting interface and in working on it. I really liked the work you did before on the extension manager and your OpenClipArt work, and if you remember I spent some time helping to get PyQt to actually work in Scribus so they'd be more viable. My only problem is with the security issue... and I think that should simply be left wide open for Python scripts, with some non-scripting safeguards such as script signing and the total inability to embed Python scripts in documents.