GSoC 2011 Tables Weekly Reports

This is a collection of weekly reports from my work with implementing tables in Scribus as part of Google Summer of Code 2011. They are kept here for future reference and can possibly be used for future design documentation. The reports are in chronological order.

Report #1

This report was sent to the scribus-dev mailing list on Sunday June 5, 2011. Here is a link to the mail.

Work Report

• Added a skeleton of a table style. This was way back in April, just to see what work is involved in creating a new type of style. The style is the most basic one imaginable. At the moment I think it has a single property for background color and styles can be managed in the SM. Also, it doesn't do inheritance yet, though that should be relatively easy to fix.
• Added class PageItem_Table, a new page item for tables. The class represents a grid of table cells by keeping a few lists with row / column geometries.
• Added ability to insert a new PageItem_Table from toolbar or Insert menu followed by mouse dragging. Just like the current "tables".
• Added API to PageItem_Table for the basic table operations
  • insert and remove rows and columns,
  • resize rows and column.
• Added support for merging cells to PageItem_Table as well as an accompanying helper class CellArea used when keeping track of areas of merged cells. The CellArea class also comes with what I think is the very first unit tests in Scribus. The tests are in tests/ and can be runned using `make test'. In the future, feel free to add more unit tests here if you think there are classes in Scribus that that are easy to unit test and could benefit from it.
• Added some rudimentary painting to PageItem_Table, just to be able to see the cells. Still a long way to go for proper table / cell painting, which is kind of non-trivial.
• As scriptability allows me to experiment with tables before any UI is done, I've added the following new scripting methods. These have been added to the old scripter, but should be easy enough to port to ScripterNG later on.
  • createTable(x, y, width, height, numRows, numColumns, ["name"])
  • getTableRows(["name"])
  • getTableColumns(["name"])
  • insertTableRows(index, numRows, ["name"])
  • removeTableRows(index, numRows, ["name"])
  • getTableRowHeight(row, ["name"])
  • setTableRowHeight(row, height, ["name"])
  • insertTableColumns(index, numColumns, ["name"])
  • removeTableColumns(index, numColumns, ["name"])
  • getTableColumnWidth(column, ["name"])
  • setTableColumnWidth(column, width, ["name"])
  • mergeTableCells(row, column, numRows, numColumns, ["name"])

Project Status

The goal in the schedule for this first week was simply "Data structures for tables". Intentionally a quite moderate goal, as I didn't know how long it would take to get up to speed. So unsurprisingly, I think I'm a little ahead of schedule. Worth noting though is that the tables are currently only skeletal in their nature; they hold no content and are simply a grid of empty cells. This is all according to plan though, and integration with text frames is scheduled for later.

Problems / Questions

Only have one quite specific question at the moment (though more will come for sure):

• How do I set a clipping rectangle on the painter when painting an item in PageItem::DrawObj_Item(...) ? Some method call to set the clipping path followed by a call to setClipPath on the painter? I looked a bit at the other items but haven't quite figured it out yet. So if someone knows off-hand how to do it that would be great.

Next Week

The goals for next week according to the schedule is

• Basic table layout with fixed column widths and mock content in cells.
• Insertion/removal of rows/columns.
• Basic drawing of table.

but as at least some of this has already been done, I think I'll start looking at other things as well. Ideas include

• Cell borders / padding.
• Add splitCells(int, int, int, int), symmetric to the current mergeCells(int, int, int, int).
• Painting of table / cells.
• Table and cell styles.
• Start looking at UI:
  • Editing through Property Palette.
  • Canvas mode for editing and selecting cells / rows / columns.
• Start looking at saving / loading (probably a bit premature).

Lots of things to pick and choose from, and if you have thoughts on which direction to go in first, I'm idle ears. Otherwise I'll just pick what feels most natural and go with it.

I don't think I want to start looking at integration of text frames just yet as I have a feeling that is going to be one of the toughest parts of the project. Besides, there are plenty of things to do before that can be done orthogonally to the integration of text frames. I'd like to have quite robust support for the other things before starting to crack that nut. Keep my back clear and don't spread myself too thin so to speak. So I think I'll stick to my schedule when it comes to this and not stick my head in that beehive until week 4 ;)

Report #2

This report was sent to the scribus-dev mailing list on Sunday June 12, 2011. Here is a link to the mail.

Work Report

• Fixed 4 small bugs in the scripting methods.
• Added a poor mans unit testing "framework" in the form a script (tests/script/test_tables.py) that automatically runs a set of test methods that exercises the table functionality and reports any regressions. The script gives output of the form

Running table tests...
	1/8: test_construction()............. Passed
	2/8: test_insert_columns()........... Passed
	3/8: test_insert_rows().............. Passed
	4/8: test_merge_cells().............. Passed
	5/8: test_remove_columns()........... Passed
	6/8: test_remove_rows().............. Passed
	7/8: test_set_column_width()......... Passed
	8/8: test_set_row_height()........... Passed
100% passed, 0 tests failed out of 8

• Added CellStyle, a new type of style for table cells. Currently it's as simple as the table style. New styles of this type can be managed in the SM. Also made sure that table/cell styles are shown below the other styles in the SM.
• Fixed the drawing problem in DrawObj_Post we were talking about.
• Added a new class TableCell for representing table cells.
• Moved some adjustment code for areas of merged cells from PageItem_Table into CellArea, thereby simplifying the code in PageItem_Table a little. This made it possible to unit test the code properly and made me find a couple of bugs that I've now fixed. The code will probably also come in handy when doing selections of cell areas later on.
• Fixed a (silly) bug when inserting a row/column before the first row/column of a table.
• (Somewhere around here made my 100:th commit, yay :)
• Changed TableCell into an explicitly shared class using QExplicitlySharedDataPointer for its data. This makes it possible for the table to easily invalidate cells that have been returned by cellAt(int, int) when rows are removed or cells merged. It also frees me from the burden of working with bare pointers to TableCells, minimizing the risk of dangling pointers.
• Various small cleanups to the table code.

In addition to the work listed above, this week I've also done quite a bit of thinking about the painting of the tables and about their internal representation. I guess like most programmers, I find that if I make a long and hard thinking about the code before writing it, it will almost write itself, whereas if I just go ahead and start coding, it's easy to code yourself into some nasty corners. As I will have limited time after the summer, I want the code to be hackable/maintainable and clear.

Project Status

The past week has seen a little bit less work from me than the last I think. This is mostly because I had that dreaded trailing exam to take care of on the Thursday. This is now taken care of and GSoC can have my undivided attention. On the schedule for the past week was as I mentioned in my last report

• Basic table layout with fixed column widths and mock content in cells.
• Insertion/removal of rows/columns.
• Basic drawing of table.

All three items are done, except I've actually removed some of the first painting code I wrote as I'm currently working on proper painting (more about this in Next Week below).

Problems / Questions

No specific questions this time. You promptly replied to me regarding that painting issue I had with DrawObj_Post(...). Thanks for that. I've added a TODO at the fix in the code about this needing some further investigation.

I do however have one non-code-related concern which turned up while writing this report; I noticed that after malex did the updating of my gsoc11tables Git repo from your internal SVN, some of my old commits started showing up twice in `git log'. As an example, my very first commit "Add a (very) rough first prototype of a table style." is now visible as both

0b17201cf816a883b80daae139f24c6a7c093939 and
0a8327820da7a88952744ac4a97314f075018869

in the Git history or gsoc11tables. The second one is the one that turned up after the merging and it has malex as commiter and me as author (Git makes a difference between the two). I'm no Git expert, but think this may be due to the pushing of SVN changes into the Git repo is done using git-svn. Ideally we would have no such duplicates, but I'm not sure how to achieve it. So; malex, do you think you could look into this? If it's not possible then it's not possible, but it's kind of annoying when trying to work out the history of things when old commits show up as if they were new. Maybe ask someone really git-wizard person about it.

Next Week

On the menu for next week according to my initial schedule is

• More advanced layout.
• Variable column widths.

But as variable column width is already done, and I already have some of the more advanced table layout features such as row/column spanning working (which was actually not in my schedule until week 7), it seems I'm still a little bit ahead. So far, it seems that I've underestimated how much work I'd be able to get done during these first two/three weeks. That said, the work ahead is plenty and I may well have underestimated the work involved in some of the later items on the schedule, so this is a good thing.

My plan of action for next week is instead to start looking at one of the items for week 4 in the schedule: "Cell formatting properties such as border and padding". In fact, this is what I'm working on right now, making some changes to the internals of the table in preparation for this. I've had a look at the quite powerful styles for lines that Scribus has, and I think they will be useful for my work. There are some tricky but interesting problems when it comes to painting borders on a table, especially regarding the order/priority of which border to paint, but also when it comes to joining of borders that meet at a corner or crossing. During my GSoC for KOffice in 2009, me and Casper Boemann were working in tandem on the tables, and he was doing a lot of the painting work, so I'm very much looking forward to taking this on as it's new territory for me.

Report #3

This report was sent to the scribus-dev mailing list on Sunday June 19, 2011. Here is a link to the mail.

Work Report

• Added a QList<QList<TableCell>> structure of cells to PageItem_Table. And made sure that cells removed from this structure are invalidated properly.
• Added the cellAt(int, int) API for retrieving cells from the table. Retrieved cells can be queried for their validity with isValid().
• Added QDebug operator<<(...) to TableCell for convenient printing of cells.
• Added correct updating of the row/column span of cells when rows/columns are removed/inserted, or when several areas of merged cells are united.
• Added a thorough internal assertValid() method to PageItem_Table which checks the consistency of the table. Also added an accompanying ASSERT_VALID macro, which compiles to nothing unless WANT_DEBUG is set.
• Added scripting methods getCellRowSpan(...) and getCellColumnSpan(...).
• Added unit test for mergeTableCells(...) to test script.
• Added scripting API for setting/getting cell border colors and widths.
• Moved cell related command out of cmdtable.h/.cpp and into their own cmdcell.h/.cpp.
• Added API for border widths/colors to PageItem_Table.
• Simplified the cellRect(int, int) and isCovered(int, int) methods in PageItem_Table by using the new cellAt(int, int) API.
• Moved styles/overview.txt into the wiki and expanded it with a little section on style property lookup.
• Added getNamedResources(...) and replaceNamedResources to CellStyle and TableStyle.
• Added two new document-wide style contexts to ScribusDoc, docTableStyles and docCellStyles, along with API that uses them:
  • const TableStyle& tableStyle(QString name)
  • const StyleSet<TableStyle>& tableStyles()
  • bool isDefaultStyle(const TableStyle& style) const
  • void redefineTableStyles(const StyleSet<TableStyle>& newStyles, bool removeUnused = false)
  • void replaceTableStyles(const QMap<QString, QString>& newNameForOld)
• Created default styles for tables/cells in ScribusDoc::init().
• Added API for handling table/cell style names to ResourceCollection.
• Added apply() and updateStylesCache() to SMTableStyle and SMCellStyle, finally allowing the SM to redefine these new styles on the document.
• Added getCellStyle()/setCellStyle(...) to scripting API.
• Added properties for border width/color to TableStyle and CellStyle.
• Changed PageItem_Table and TableCell to use an internal style for storing local properties. The style has the new style context added to ScribusDoc as context, and setting a named style on a cell or table will set the parent style of the internal style.
• Added API and scripting API for background color to tables and cells.
• Added missing update(...) functions to TableStyle and CellStyle.
• Added getTableStyle()/setTableStyle(...) to scripting API.
• Added some initial painting of cell / table backgrounds. Color is taken from style and direct formatting also works.
• Various documentation fixes, bug fixes and cleanups.

Project Status

On the schedule for this past week was

• More advanced layout.
• Variable column widths.

But as those were already done, I decided last week that this week I'd instead start working on proper painting of cell borders. However, despite getting a lot of things done, it seems my schedule has finally caught up with me. You could say that I'm now on schedule instead of ahead of it.

The main reason why I'm no longer staying ahead is that it's been more work than anticipated in bringing the styles up to snuff. A lot of legwork to get them integrated into the the document and updates SM<-->ScribusDoc working.

Things are pretty much ready now though. New properties can now easily be added and the tables and cells will pick them up. Work on cell/table border painting can now begin.

Problems / Questions

No specific questions this time either. I had some during the week, but those were resolved over e-mail/IRC. (Thanks to Craig/avox!).

Next Week

In the timeline for next week it reads

• Cell formatting properties such as border and padding.
• Cell content layout; use Scribus' text frames to populate cells.

And for the first time since my first week, I'll actually be doing what the schedule says :) I'll work on the painting of tables/cells. Solid backgrounds are actually already painted, as of a commit some hour ago. Hopefully I'll also be able to also start working on bringing Scribus' text frames into the game. I'm sure that will be a lot of work, which is why it's in my schedule for the week after next as well.

Report #4

This report was sent to the scribus-dev mailing list on Monday June 27, 2011. Here is a link to the mail.

Work Report

• Some small cleanups to insertRows()/-Columns() and removeRows()/-Columns().
• Added a new empty page "Table" to the PP.
• Added properties for table border model and border drawing options to the table style / table.
• Drafted a design of how borders could be collected by "border collectors" inheriting from a common base class and started working on it.
• Added a drawLine(...) taking QPointF to ScPainter.
• Realized my initial design mentioned above was probably unnecessarily complex / over-designed and abandoned it.
• Added a PageItem_Table::collapseBorders(...) which does primitive collapsing of two borders.
• Added four functions to PageItem_Table for getting the correctly collapsed borders of a cell.
• Added PageItem_Table::drawBorders(...) for painting a list of borders.
• Added PageItem_Table::drawTableCollapsed which paints the entire table with collapsed borders. It works by making one iteration across the cells of the table, painting cell backgrounds and collecting borders as it goes. It then paints the collected horizontal and vertical borders in the correct order in a second pass.
• Switched to using QRectF / QPointF instead of FRect / FPoint.
• Added PageItem_Table::adjustBorderJoins(...) which adjusts the endpoints of the four borders surrounding a cell according to the border drawing order (horizontal first or vertical first).
• Added setTableBorderDrawingOptions(...) to scripter API.

Project Status

On the schedule for this past week was

• Cell formatting properties such as border and padding.
• Cell content layout; use Scribus' text frames to populate cells.

And quite frankly (and unfortunately) I haven't managed to do half of the things I wanted to. The reasons for this are several and mostly has to do with bad planning on my part, so I take full responsibility for all of them.

Firstly, we had visitors from Singapore coming in on Tue-Wed, and the plan was that Hanna would take care of showing them around town et.c., as she was supposed to have those days off. But they called her in for work in the last minute, leaving me to take care of the guests. Secondly, for Midsummer celebrations we were as usual going up to some friend's summer house up in the northwest. Before I left I made sure that tethering using my Android phone would work in Linux, and looked up the 3G coverage up there, as it's quite a remote location. My hope was to get some hacking done while up there. Tethering worked out of the box and the coverage map I looked said it would be OK. Unfortunately it was only good for certain providers, and coverage was horrible (even regular calls were a bit shaky for me). Finally, as the tickets for the train we were going to take up there on Friday rapidly grew outrageously expensive, we had to book tickets for early Thursday instead. This all meant that when I got home late Saturday night, I hadn't gotten much work done at all. Hence the meagre list in the Work Report this time around. It's a combination of unrealistic planning on my part and some bad circumstances.

But enough about that. The project status right now is that basic painting of solid collapsed borders is pretty much done. The approach I finally settled on seems to work out OK, though some things could of course be improved.

Problems / Questions

Nothing at the moment. One thing that gave me a little surprise was that I realized e.g. FRect::right() will not return left() + width() but left() + width() - 1 (just like that infamous "bug" QRect has for historical reasons). I've since switched to QRectF.

Next Week

In the timeline for next week is

• More integration of text frames. Fixing of issues/road bumps.
• UI: Row/column selection/insertion/removal/resizing.

How appropriate that I managed to get the words "road bumps" in there for this week :)

On a more serious note, what I'll be working hard on during next week is of course getting back on schedule. First I'll fix any remaining issues with the basic border drawing I have now. This means specifically

• Add a vertical and horizontal offset to the table layout. The basic table grid should actually be offset by (max(left borders of first column)/2, max(top borders of first row)/2)
• Add cell padding (this is easy, it's just a few more properties on the cells).

After that I should have content bounding rectangles for the cells, and can finally turn my head to integrating Scribus' text frames. As the schedule also mentions UI for next week, I'll try to at least start looking at canvas modes and cell selections.

Report #5

This report was sent to the scribus-dev mailing list on Tuesday July 5, 2011. Here is a link to the mail.

Work Report

• Fixed the API docs of Scribus to work with Doxygen 1.7+.
  • Some HTML elements had changed, so the CSS was updated to match.
  • Updated Doxyfile with `doxygen -u'.
  • Added *.js and *.repository to `clean' target as those are generated nowadays. No *.ttf seems to be generated anymore, but I kept them in to stay compatible with older Doxygen.
  • Enabled JAVADOC_AUTOBRIEF (OK:ed by avox on IRC).

The above is not directly related to my project, but as I try to document all code I'm writing I wanted to see what it looks like by generating it locally. Anyway, it was 15-20 min of work. Moving on, I also

• Added class TableBorderLine which represents a single line in a table/cell border. Perhaps it can be scrapped in favor of the SingleLine class in scribusstructs.h as they are very similar, but I'm keeping it for now.
• Added class TableBorder which represents a border along one side of a table/cell. It is essentially a list of TableBorderLine with a convenience function bool joinsWith(const TableBorder& other) const; to check if it can be joined with another border. A TableBorder with no TableBorderLines in it is considered a null border. These are used for convenience during painting to represent the absence of a border.
• Updated the scripting API and table/cell styles to use the new border system instead of individually set border color/width/et.c. properties.
• Added abstract base class TablePainter representing a painter that can paint a PageItem_Table. The class has a single pure virtual function void paintTable(ScPainter*) in which subclasses should do their work.
• Added class CollapsedTablePainter, an implementation of TablePainter that can paint a table with borders collapsed and joined at intersections.
• Modified PageItem_Table to hold a pointer to a TablePainter and paint itself using this painter in the DrawObj_Item(...) function.
• Added three utility functions, kept in a new TableUtils namespace for easy unit testing. The functions are used by CollapsedTablePainter when collapsing and joining borders. They are
  • collapseBorders(...) which given two borders returns them collapsed,
  • joinHorizontal(...) which given a horizontal table border and its adjoining borders will return the adjustments necessary for joining the border,
  • joinVertical(...) which given a vertical table border and its adjoining borders will return the adjustments necessary for joining the border,
• Added columnPosition(int) and rowPosition(int) API to PageItem_Table.
• Added an QString asString() const; to CellStyle for easy printing.

In addition to just listing the changes like this, I feel I need to elaborate a bit on what has taken such time. I did explain it a little bit in an earlier mid-week report. The problem has been a hard one for mainly two reasons:

• Each cell may have N neighboring cells along one of its edges. The objective of the painting algorithm is to iterate over all cell edges in the table, and for each cell edge, iterate over all shared border sections. It must then figure out all (possibly six) cells surrounding the border section to be painted, along with the collapsed borders between them (or between them and the table). When that is done, the border section (or rather, the coordinates at which to paint the border section) needs to be adjusted for joining. This is delegated to the joining algorithm.

• There are 41 one possible join cases (that I've identified at least). They can be seen in the attached picture, which I finally had to draw to sort it out in my head. The objective of the joining algorithm is then to, given a border section along with the other border sections coming in to meet it at its start and end point, return adjusted start and end points as well as offsets for the individual border lines in order to make a proper join.

On top of this is the question of painting order. The joining algorithm must be necessity make an assumption about painting order; either horizontal borders should be painted on top for the algorithm to work, or vertical should be painted on top.

The joining algorithm I've written assumes that all horizontal borders in the table are painted on top of vertical ones, which means two iterations across the cells. I've been unable to find an algorithm which makes no assumption about the painting order. I don't think there is any actually, and iteration across the table cells should be fast enough. Perhaps in the future optimizations could be made, but then I think they should rather be done to the internal structures of the table to make cell accesses fast, rather than convoluting the joining algorithm.

All in all, a tough nut to crack, and there are currently the following two small problems with the painting/joining left:

1. The joining algorithm does not handle some cases properly.
2. The painting algoritm sometimes mistakenly identifies a neighboring border as a null border.

For 1), I feel that I need a break from this problem to not go insane, and will let it go for now. The code is completely isolated in TableUtils::joinHorizontal(...) and TableUtils::joinVertical(...) and can be improved on later. I welcome anyone who likes puzzles to look at the picture with the 41 join cases and see if they can find an optimal algorithm that covers all cases with a minimal amount of code.

For 2) I know the reason and how to fix it, it's just that I haven't gotten around to it. In short, what happens is that instead of collapsing with the table border, it's requesting a cell outside the table, which will return an invalid cell with a null border, and collapses with that.

As I've intentionally kept the collapsing and joining algorithms as separate functions in the TableUtils namespace, what remains is to write unit tests that cover all the 41 cases.

Project Status

I'm still behind schedule by at least one week. The reason I decided to put all my efforts into getting this painting working correctly is I didn't want to leave it unfinished, or leave a design that would later come back and bite me when trying to paint complex borders.

I had no idea it would be such much hard work. I think I've used up all my sketchbooks on this problem, and lost some sleep over it. I feel now that, to not lose my mind, I really need to get on with some of the other problems, such as UI and integration with text frames.

I've looked ahead a bit on my schedule, and found that

Week 6:

• Wrapping up for mid-term evaluation.
• Fix bugs/finish up what I have so far.

Week 7:

• Row and column spanning.

Week 8:

• Table/row/column/cell styles.

Since row and column spanning is done, and the styles are in a pretty good shape (they are there, and new properties can be added removed easily), I feel that there's still a good opportunity for me to catch up.

Problems / Questions

My biggest question right now is:

• What should be the general approach for integration of text frames? The table will hold a set of text frames for its cells, but then? The user should be prohibited from doing certain things to the text frames, such as selecting and resizing them. This should be handled by the table. In a way they will be similar to locked text frames, until the user decides he wants to edit their content. Should I go for an approach where I add a bool isTableCellContent() or similar to PageItem_TextFrame? Or should PageItem_TextFrame even be inherited, creating a new item for cell content? I'm open to all ideas, so if you happen to have given this problem some thought, please share, because frankly I haven't (yet).

Next Week

I'll yet again be working on catching up with my schedule, which means working on integration of text frames and basic UI. Since mid-term evaluation is next week, perhaps I should also write some kind of mid-term report. What do you think Craig, do you think its worthwhile or just a waste of coding time? Perhaps you feel up to date enough with my status. I also really should get around to making another blog post showing the new features, as I've only made one post so far.

Good morning,

Here comes my 6:th report [1], along with a little screencast showing the bits of new UI I've added:

http://www.nya-doxa.se/elvis/gsoc2011/week6_table_pp_editing.ogv

Of course none of it is set in stone, so please comment.

Cheers, Elvis

Report #6

This report was sent to the scribus-dev mailing list on Monday July 11, 2011. Here is a link to the mail.

Work Report

• Changed logic if TableBorder::joinsWith(..); null borders can never join.
• Finally fixed the joining algorithm. It is now 100% according to spec. I had actually missed two cases of possible joins. Any more work to be done on the joining algorithm are just aesthetic adjustments for the more obscure cases (e.g. four borders of different width meeting at a cross et.c.).
• Added a test script that tests the 43 join cases (see attached screenshot).
• Changed grid lines to be painted in a separate pass. This is needed as they need to be on top of everything else. TODO is to add a setting for turning them on/off.
• Fixed a bug in grid line painting on cells that span multiple rows/columns.
• Fixed a similar bug in cell fill painting.
• Fixed an off-by-one bug in the calculation of border segment interval.
• Fixed the bugs in the neighboring cell determination I was talking about in my last report. I've since realized there's another bug there relating to spanning cells, but that one should be easy to fix.
• Added a new widget TableSideSelector for selecting which sides of a table/cell to apply border properties to.
• Cleaned up and extended the API of TableBorder a bit, and made sure that border lines are always in descending order by width.
• Initialized borders properly in default table/cell styles.
• Added support to PropertiesPalette_Table for editing table borders and fill (using the new TableSideSelector widget).

Project Status

In the schedule for week 6 was

• Wrapping up for mid-term evaluation.
• Fix bugs/finish up what I have so far.

But my goal as stated in the last report was catching up with the schedule, so I've both been working on fixing bugs and on the UI. Unfortunately I've still not had time to do any text frame integration work, apart from some discussions on the ML. But as I said before, there are some gaps in the schedule coming up and I'm still hopeful that I'll catch up completely.

Problems / Questions

During the UI work a couple of things popped up:

Question 1

• When the item selection changes, PropertyPalette::handleSelectionChanged() is called, which in turn calls handleSelectionChanged() on my table property palette. In there, I take care of setting up stuff for editing of the selected item. Among other things, I reset the new table side selection widget to have all four sides selected. This is the initial state of the widget when editing a newly selected item.
• After changing a property on the table from the PP, I call update() on the item to request an update, which I guess is the correct way to do it?

The problem is that the call to update() will in the end result in handleSelectionChanged() being called. I've discovered that this is true for all items; when editing a property of an item in the PP, handleSelectionChanged() will be called, even though no selection change has really occurred, just a change in an item property. This has the following consequence

• Select a table.
• Select just the top side in the new side selector widget.
• Change the border width using the spinbox.
• Ouch! Since handleSelectionChanged() was called, the palette assumed the selection changed and therefore reset the side selector widget to have all four sides selected.

At the moment I work around this by checking in handleSelectionChanged() if the selection really did change (pointer comparison to previously selected item). So my questions regarding this is

1. Is update() the correct function to call to request an update after changing a property on an item?
2. Is it desired behavior that PropertyPalette::handleSelectionChanged() is always called when update() is called on one of the items in the document? The selection didn't change..
3. Is my workaround above OK or is there a better way?

Question 2

I've now finally hit some problems relating to painting outside of the table frame (before I didn't have any artifacts because of this, but now they've started to appear sometimes when changing the border through UI).

So how should we handle this? Should the top/left side of the table always stay inside the table frame (e.g. be offset by max(top_borders)/2 in the Y direction and max(left_borders)/2 in the X direction) or not?

Next Week

The schedule for next week simply says

• Row and column spanning.

which is pretty much finished, so I think I'll try to keep working on UI to get some steam going on that. I'll finish off editing of the supported table features in the PP and then do the same for the SM. Then I'll start looking at cell selections / canvas modes and the editing of cell properties. I think I'm reconsidering the approach I've taken with using a QListWidget for the border lines, and I'll probably turn it into a proper model/view instead.

Since the mid-term evaluation deadline is on Friday, I'll also write up a short summary report of the first 6.5 weeks of the project, and maybe make another blog post showing off the new UI.

Report #7

This report was sent to the scribus-dev mailing list on Wednesday July 20, 2011. Here is a link to the mail.

Work Report

• Changed PageItem_Table to declare its properties using Q_PROPERTY and to emit a changed() signal when the table changes.
• Added unset*() functions for the properties in PageItem_Table.
• Added functions for getting the maximum border widths in effect along the four sides of a table.
• Used the above functions to translate the painter. The table is now always painted inside its frame at the top/left.
• Changed PageItem_Table::adjustToFrame() to take the borders around the table into account.
• Fixed a bug in ScribusDoc where the PoLine (point array for the frame geometry) wasn't set correctly on the table.
• Fixed painting to clip to the frame.
• Added a PageItem_Table::resize(...) function to PageItem_Table. The function supports two resize strategies; Equal and Proportional. Equal will give each row/column and equal amount of the space to be distributed, Proportional will give them space proportional to their current width/height. When shrinking a table, the function will never shrink it so that a row or column goes below the new limits MinimumRowHeight and MinimumColumnWidth, respectively.
• Updated unit tests in Python script to take the changed behavior of adjustToFrame into account.
• Renamed adjustToFrame() to adjustTableToFrame() and added a symmetric adjustFrameToTable() function.
• Added UI support for "Adjust Frame to Table" and "Adjust Table to Frame", similar to how it's done for image items: These new actions can now be performed on one or more selected tables using context menu or application menu actions.
• Fixed a crash when copy/pasting a table. The fix is temporary and simply makes sure that an item of the right type is inserted on paste. Real copy paste support needs serializing of the table to Scribus file format though, which will come later.
• Fixed the (hopefully) last lingering bugs in the border collection code in CollapsedTablePainter.
• Did some aesthetic improvements to the handling of case 14 and 15 in the border joining: When four borders meet at a cross and they can join pairwise, it looks much better if the thicker of the two pairs win the conflict. So we separate the two cases 14 and 15 into 14A, 14B, 15A and 15B. I think our handling of these cases are now more graceful than InDesign's. The picture of all the cases on the wiki has been updated, as well as the test script that tests them all.
• Various doc fixes and cleanups.
• Wrote a short mid-term report for the mid-term evaluation.

Project Status

On my schedule for the past week was

• Row and column spanning.

but since that was already done, I decided last week that I would keep working on the UI, and while I have done some work on that, my illness really put a dent in my productivity this week, and I didn't get as much done as I wanted.

Problems / Questions

No specific questions at the moment.

Next Week

The schedule for next week says

• Table/row/column/cell styles.

and I guess I'll be doing some work on their UI.

However, with the mid-term evaluations done, even though there are still quite a few weeks left, it's starting to feel like the project starting to come to a close.

I think it's very important now that I start to really focus on what is missing to make what I have work like proper tables. To make it possible to actually create and edit tables, with content. To that effect, in the next week I propose that I:

• Keep working on the UI bits that are essential to basic tables support.
• Really get my ass out of the wagon and start experimenting with integration of text frames and story text in cells, following the discussions we had here on the ML.

Report #8 and #9 - Double Report

This report was sent to the scribus-dev mailing list on Monday Aug 1, 2011. Here is a link to the mail.

Work Report

• Fixed the old code for painting the overflow marker and moved it up to PageItem.
• Added isOverflowing() API to PageItem_Table and added painting of the overflow marker to the table painting code.
• Added cellAt(QPointF, QPointF) API to PageItem_Table.
• Added the skeletons of a canvas mode for editing tables.
• Added hitTest(QPointF point, qreal threshold) to PageItem_Table. Given a point in canvas coordinates, this function will return a PageItem_Table::Handle object that describes what was hit.
• Added support for resizing rows and columns interactively.
• Added support for resizing the entire table interactively.
• Lots of minor cleanups, API improvements and a few bugfixes along the way.

Project Status

On my schedule for the past week was work on the styles

• Table/row/column/cell styles.
• UI: Direct formatting.
• UI: Integration of styles in Style Manager.

but instead I decided to work the on-canvas manipulation of the tables. One of the reasons I did this was that I feel that I've already been touching the PP and SM code and feel that it is mostly mundane work, while I hadn't looked at the canvas modes/gesture code at all. So I felt that I needed to get my hands dirty there as well, to get a good feel for how much work is involved.

Getting the canvas modes up and running turned out to be quite easy -- the majority of the work on them was done in the past two days. I feel confident now that I'll manage to finish the remaining ones (cell selections and I guess context menus for adding/removing rows/columns, merging cells et.c.).

The unfortunate thing that happened since my last report was that I fell very ill with what I first thought was just a flu. But after a doctors visit when the fever went up, it was discovered to be a more serious infection, one which I'm now on antibiotics for. But eventhough I'm still on the meds, I feel a lot better now and can work full time again. My work on the project did suffer a bit though.

Problems / Questions

1. As mentioned in another thread on the ML, I can't seem to get the guides and grid shown. For some reason they are always invisible for me, even though I have "Show Grid" and "Show Guides" on. Any ideas? I've tried changing their colors, putting the in foreground/background as well as wiping ~/.scribus. If anyone is testing my repo, it would be great if you could check. I really hope I didn't break anything and that it's just some weird graphics issue or something (I'm on nvidia). But I don't think I've touched any of those code paths.
2. When I wanted to have the selection of the table painted similar to that of a text frame when it is edited (e.g. just a red rect, no handles), I found that I could do this by calling CanvasMode::commonDrawControls(). However, I discovered something that I think is kind of nasty; if you take a look at CanvasMode::drawSelection(), which is called by commonDrawControls(), you'll see that it performs transformations on the painter without save()ing and restore()ing it properly. I discovered this since I was doing my own transformation of the painter. Okay I thought, I'll just fix that, so I added save() and restore(). I then discovered that a lot of the current items rely on this behavior. I really doubt this was intended? So it is a bit of a work to make a proper fix. At the moment I worked around it by guarding my call to CanvasMode::commonDrawControls() with save()/restore() myself, e.g. p->save(); commonDrawControls(); p->restore(); .. But this is a hack. Just thought I'd give a heads up on this.
3. Minor question; I guess c++0x is still off limits for Scribus code?
4. InDesign draws a simply blue grid while a table is resized, without any respect for cell spans/column spans. In my implementation, the blue grid will respect row span / column span. Which one do we want? I guess ID did it like this because its simpler (and possibly slightly faster).
5. Now that I've worked a bit with the canvas mode code, I've gotten this idea in my head for the editing of cell text content; what if I create a canvas gesture that simply acts as a proxy to an instance of the real text editing canvas mode (CanvasMode_Edit), which would be set up to edit the text frame of the cell being edited? Do you think such an approach could work? This proxy gesture would forward all events (possibly intercepting some of them for cell navigation). It might even be activated when a cell is hovered with the mouse, which would save us from multiple layers of modes having to be activated by the user to be able to edit cell content (But for a start, I would activate it on double click on a cell). The approach has a slightly hacky vibe to it, but if it could work I think it would be great as all the functionality of the existing text editing canvas mode could be leveraged. And what is really the alternative? Someone (was it avox?) mentioned in another thread that one should simply forward mouse presses and use insertText() in the keyboard event or something to that effect? Perhaps this is what was meant...

Anyway, too many questions already for one report :)

Next Week

It feels like this time, this is the most important section of the report. In my mind, there are four main areas I need to focus hard on in these last weeks of the project. In order of priority, they are

1. Finish off the canvas modes / gestures I'm working on now. This includes
   1. Finding a good way for the table to represent cell selections. I'm thinking a new CellSelection class that simply keeps a QSet of the selected cells, with convenience functions such as (de)selectCell(int, int), toggleCell(int, int), (de)selectRow(int), (de)selectColumn(int), selectTable(), clear() et.c.
   2. Adding some form of context menu with actions such as "Insert Rows...", "Insert Columns...", "Merge Cells" et.c. I plan to look at this tomorrow.
   3. Add the possibility to move the boundary between two rows or columns independently of the other boundaries.
   4. Add some form of tooltip popup that shows the numeric information when resizing a row/column.
2. Integrate text frames. Each cell should store a text frame and an associated story text, and the table layout code will have to be extended to adjust these as part of its layout procedure.
3. Finish off PP and SM UI for direct editing and editing of styles. This includes among other things a way to set column widths / row heights numerically.
4. Loading and saving (Low priority since I believe it will mostly be mundane work)

If I have many days like today, getting lots of stuff done, I believe this to be possible before the firm pencils down date on Aug 22. And as I said in my proposal, I'll keep working full time on the project for another week after that, until school starts (and of course after that as well, to the best of my abilities).

Report #10

This report was sent to the scribus-dev mailing list on Tuesday Aug 9, 2011. Here is a link to the mail.

Work Report

• Fixed a bug in the painting of page grid, baseline grid, guides and margins. The old code assumed that ScPainter would stroke even though stroke mode was set to None. Unrelated to table project but was uncovered by a change I made to ScPainter.
• Simplified the table hit test a bit.
• Display tooltips with width/height when during resizing of rows/columns/table.
• Gave the table related canvas gestures a common base class to avoid some code duplication.
• Added support for selecting cells. Individual cells may be selected by clicking them, rectangular areas by click dragging. The selection is painted as a blue-tinted overlay. The new API for selecting cells on the table is:
  • void selectCell(int row, int column);
  • void selectCells(int startRow, int startColumn, int endRow, int endColumn);
  • QSet<TableCell> selection() const;
  • void clearSelection();
• Added an optimization to the painting of the table outline during resizing. I'd say it's now ~10x faster.
• Added support for resizing rows/column by moving the boundaries between them independently. This mode is entered by holding down Shift while resizing a row/column.
• Added padding properties to cell and cell style, and scripting methods to set them.
• Started working on cell text content. It's now possible to set the content on a cell using setText() (scripter API also available). The text of the cell is represented by a PageItem_TextFrame, and is painted as part of normal table painting. The calculation of the content rectangle occupied by the text frame takes all the borders along the cell edges as well as paddings into account, and is updated if any borders/paddings are changed, if a style is changed or if a style is set/unset.

Project Status

I don't really feel there's a reason to bring up what was in my schedule for this past week, but for brevity, here it goes anyway:

• Scripting API.
• CSV and/or ODT import.

Funny how "Scripting API" is in there, since it has a been a corner stone for me when it comes to testing throughout the project :)

Jokes aside, I feel pretty good about the project status. I was pleasantly surprised that getting the text frames painted went so fast (except that overflowing problem of course). Now it remains to see what kind of problems I'll hit with editing. But I'm staying positive.

There are only a couple of weeks left until the official pencils down date, and after that I will work another week full time before the school starts.

Problems / Questions

Nothing at the moment. I thought I had run into a wall with that overflowing problem, but it turned out to be just a oneliner fix after all.

Next Week

On the schedule for week 11 (this week) is:

• Wrapping up for final evaluation.
• Fix bugs/finish up what I have so far.

This means I'll try to do as many of the following tasks as possible, in order of priority:

• Basic cell text editing.
• Context menu.
• PP UI.
• SM UI.
• Saving / Loading.
• Painting for print output.

Report #11

This report was sent to the scribus-dev mailing list on Sunday Aug 14, 2011. Here is a link to the mail.

Work Report

• Save/restore global shortcut actions when entering/leaving table edit mode.
• Added the concept of an "active cell" to PageItem_Table. A table always has an active cell, which may be set/get using setActiveCell(...)/activeCell(). This is the cell that will receive keyboard input during table editing, and the cell which table operations will operate on when there's no cell selection.
• Added support for typing text into cells. Typing is handled by passing on key press events to the handleModeEditKey(...) function of the text frame of the active cell.
• Added drawing of a blinking cursor while in table edit mode. The blinking behaves like it does for normal text frames, making a long blink on initial cell activation, as well as when Home/End/PgUp/PgDown has been pressed.
• Changed the PageItem_TextFrame to paint text selection while in table edit mode, not just normal edit mode.
• Added support for positioning the text cursor in table cells with the mouse. This required modifications to ScribusView::slotSetCurs(...).
• Added support for selecting text in cells with the mouse. The double click behavior is the same as for normal text frames; double clicking selects a word, with the Ctrl modifier a paragraph is selected and with Ctrl+Shift modifiers a range of paragraphs is selected. If the mouse goes outside one cell and into another during text selection, the cell selection gesture is started.
• Added support for clicking outside table with mouse to leave the table edit mode.
• Stop the cell selection gesture when just clicking with the mouse.
• Position the cursor correctly in cell on initial table edit mode activation.

Project Status

My goal for this week was to get as much of the following done:

• Basic cell text editing.
• Context menu.
• PP UI.
• SM UI.
• Saving / Loading.
• Painting for print output.

I'd say that the first item on the list is now done. There remains some minor details such as being able to go from text editing of one table to another (or to a text frame) with just a single click. This should be simple to fix, but is not an essential feature.

Problems / Questions

I have one question regarding the destruction order of removed items vs. styles which is currently causing a crash on application exit, but I've sent that in a separate mail to the list. Apart from that, no pressing questions right now.

Next Week

I'll now move on to the other items on my hit list above, starting with the context menu.

Report #12

This report was sent to the scribus-dev mailing list on Sunday Aug 21, 2011. Here is a link to the mail.

Work Report

• Fixed a crash when deleting a table and then quitting the app.
• Speedup of row/column insertions by preventing call to checkChanges() on cell text frames.
• Added time measurements to the table test script. It now shows the time for each test, as well as the total test time.
• Declared TableCell, TableBorderLine and CellArea as Q_MOVABLE_TYPE. This allows the Qt containers to use fast memcpy() instead of calling constructors when moving objects of these types.
• Added a "Table" submenu to the "Item" menu.
• Added support for inserting rows and columns from menu.
• Added convenience API selectedRows() and selectedColumns() to PageItem_Table and renamed selection() to selectedCells().
• Added support for deleting rows and columns from menu.
• Added support for merging cells from the menu.
• Added support for setting row heights and column widths from menu.
• Added support for distributing row heights and columns widths evenly from the menu.
• Fixed the problem with the cursor being initially invisible when activating text frames (Craig said you might want this fix in 1.4).
• Added support for navigating between cells with Alt+Left/Right/Up/Down. This required the table to be given the concept of a "logical position" in addition to the "active cell", since you want to be able to navigate through a cell spanning multiple rows and/or columns and end up in the right place. The new API of PageItem_Table for navigation between cells is moveLeft(), moveRight(), moveUp() and moveDown(). These functions will move the "logical position" and set the active cell appropriately.
• Fixed a crash when deleting a series of rows or columns that contain an entire area of merged cells.
• Added a context menu that is shown on right click when editing a table. Some of the actions (the ones that are applicable) are also shown in the regular context menu for Scribus items.
• Added some icons for table actions from the Oxygen project. Nuno thought it would be a good idea, and he's a much better icons designer than I am :)
• Fixed a bug in the painting of the table outline during resizing.
• Various cleanups and doc fixes.

Project Status

Since this is the last official report, this time I'll make a little richer report on what is possible with tables so far.

Table Operations

All the basic operations on the structure of table such as inserting, deleting and resizing rows and columns, resizing the table as well as merging cells are now supported through the UI. In addition to that there are a few convenience operations such as adjusting the table to the frame and vice versa, and distributing a range of rows or columns evenly.

Cell Text Content

The text content of cells may be edited on-canvas, and navigation between cells is possible with both the mouse and keyboard. Neither direct formatting of the text in the cells nor applying paragraph or character styles is supported yet (see "Next Week" below).

Borders / Paddings

Borders may be set on each side of a cell, or on each side of a table. There is support for multiline borders, and the borders will be properly collapsed and joined at intersections. Some finetuning of the collapsing algorithm may be desirable though. Paddings may be set on each side of a cell.

There is rudimentary support for editing the borders of the table in the Property Palette, but apart from that, the support for borders and paddings are only accessible through scripter methods, so more work is needed on the Property Palette UI (see "Next Week" below).

Styles

There is support for table and cell styles, however the UI for editing the styles in the Styles Manager is largely lacking (see "Next Week" below).

I've kept out some details, but I believe that's it. This is the status of the project as of today.

Problems / Questions

No questions this time. I do have one for the future regarding usage of Boost in Scribus. But that one is in a separate mail.

Next Week

Looking at my brief TODO list from my last report:

• Basic cell text editing. [DONE]
• Context menu. [DONE]
• PP UI.
• SM UI.
• Saving / Loading.
• Painting for print output.

I now consider the first two done, even if there are some small details left. For this last week of full time work that I have left, I plan on working on the third and fourth item in the list: PP and SM UI. More specifically, I plan to:

Properties Palette

• Support direct formatting of cell text, as well as applying paragraph and character styles.
• Improve the "Table" property palette.
• Add a "Cell" property palette for editing of cell properties.

Style Manager

• Add UI for editing table style.
• Add UI for editing cell style.

If there is time left (probably unlikely), I'll also start work on the last two items on my list: saving / loading and painting for print output.

User Centered Goal

Since tables support is now starting to come together quite nicely, I'd also like to put a user centered twist on this week's "Next Week" section: Attached to this mail you will find two screenshots. One of them shows a simple table design I did in InDesign. The other screenshot shows what I was able to accomplish after some work in Scribus. The user centered goal for this last week is that by Sunday, I want to be able to make a screencast of me creating the design as shown in the InDesign screenshot, in Scribus. I want to be able to do this in less than 3 minutes and I want to be able to do it without having to bring up the scripter console.

(Wiki Note: I'm adding the screenshots mentioned above here.)

The "trial design" discussed above as created in InDesign.

The "trial design" discussed above as created in Scribus.