Hi there,

it seems some kind of strategy is needed concerning various files for storing MW4 settings. There were quite a lot of those before (.mwproj, .mwsr, .mwcfg, .mwleg, .lbl, some others?). Now when a new functionality was added to ocx, there is a problem with compatibility of those files.

Let's admit that it was a fault of planning not to consider these issues in the first place. Now we simply HAVE TO deal with it.

So, please, post all the ideas/remarks/bug reports on the topic here. I'll describe possible solutions for the 2 issues now.


1. Layer properties file (.mwlyr, .mwsr, etc).

There are 2 versions of files to hold layer properties:

- new .mwlyr;
- old .mwsr (.mwleg, .lbl).

The idea (as far as I remember not mine) for adding .mwlyr was:
- to store all the layer properties in a single file (.mwsr, .lbl, .mwleg);
- to open .mwlyr from OpenFileDialog;
- to store several .mwlyr files for the same layer (to be able to add the layer to the same project several times with different settings);

A question: do we need .mwlyr file?

The suggestions I see [bugs.mapwindow.org] (last post) is to open .mwlyr associated with layer automatically in the same manner as .mwsr, .mwleg are treated. Then why not to create .mwsr with "Version = 2" attribute and stop doing excessive work?

.mwlyr problem I see: in case there are several .mwlyr files for the same layer exist, there is serious problem with naming. If name is different from the data file (shapefile for example) it's difficult to remember what .mwlyr file actually targets. Most likely a garbage from the forgotten .mwlyr files will be accumulating in folders as a result which won't give any pleasure to users.

SUGGESTION:
I'd rather treat .mwlyr like this:
- layer properties file should be saved under the same name as data file automatically;
- if several .mwlyr files are saved, an index should be added to the name automatically;
- .mwlyr files shouldn't be opened from the OpenFileDialog;
- when opening a data file with associated .mwlyr files a window will be shown:

"Do you want to load one of the predefined options?"
A. Load from .mwlyr 1
B. Load from .mwlyr 2,
...
C. Load with random options;

Layer files must have some embedded description of course for user to choose. Also it's possible to show preview in the same window, where new layer with particular options will be visualized.

Some GUI should be added to enable deleting/annotating different layer settings (.mwlyr files). It can be done in a separate tab of the Symbology window, for example.

For such behavior actually there is no need to use new extention, but to use old .mwsr. As for .mwlyr file I'd rather limit those to the non-filebased datasources (like databases, online servers, etc), support of which will be hopefully added to MW4.

What do you think?


2. Improving serialization.

Presently for serializing properties to .mwlyr file .NET code based upon System.Reflection is used. It's written in VB.NET in MW4 project. The deficiency of the approach is obvious. In the best case it will be used by .NET users (if they will be able to find it). But we know there is need to store properties in almost every MapWinGis application. Also it would be quite handy to support .mwlyr file in ocx natively. Then a user could set necessary properties in MW4 and then to load the layer in own application.

So currently I'm working on the moving serialization code to ocx. There will be methods like Serialize/Deserialize in every class where it is applicable (I'm half way through this). I'm using GDAL utility functions to deal with XML.

Then I plan to add methods for reading/writing .mwlyr file from ocx (the properties that are specific to MapWindow/Legend control will be ignored). MW4 will use ocx serialization for .mwlyr and add the specific things (like group in the legend).

Also I'd like to restore Map.State property, where settings for all layers will be dumped in a single XML file.

If there are remarks on the functionality, please, let me know.

Thanks,
Sergei

Sergei,

I am the one that suggested to create a layer file. I thought of it when using ArcGIS v9. ArcGIS can add the same shapefile twice and make both layers appear differently.
For example:
Open the counties shapefile and give them a color based on their states. So all counties in a state have the same color. Set the dynamic visibility to hide this layer when zoomed in.
Open the same shapefile again, now give every county a color of its own. Set the dyn. visibility to only show this layer when zoomed in (and the other layer is off).
The same must be possible with labels and charts.
The above example is with a polygon shapefile, but the same is for points and lines shapefiles of course.
For a point shapefile you can imagine you open the shapefile twice and set the size and colors of the points in both cases to a different attribute value.
I believe this is only possible if you can open the shapefile twice, but have separate label and color settings --> a layer file.
If the above scenario can be made in a different manner it is OK by me as well.
If you want the mwlyr file and shapefile be related through the filename, the above scenarios wont work anymore and the mwlyr file has no use since it will work like the current lbl and mwsr files.

Like Sergei I also want to use the mwlyr file for databases, WMS/WFS/TMS services and such. So we definitely need to keep this option.

I'm OK with moving the serialization to the ocx. It would be nice if the ocx also has knowledge about the project file. Then we can create a new project file using the ocx as well (much requested).

Thanks,
Paul

--
Don't forget to read the new documentation: www.mapwindow.org/documentation/mapwingis4.8
Join us Google+: MapWindow GIS Google+ Community
Join the MapWindow Group on LinkedIn! LinkedIn - MapWindow Group

Download the latest beta installer at:
tinyurl.com/mwMonthly 32-Bit
tinyurl.com/mwMonthlyx64 64-Bit
Follow me on Twitter MapWindow_nl to read when a new installer is published.

---
Paul Meems
The Netherlands
[www.bontepaarden.nl]
Release manager, configuration manager and
forum moderator of MapWindow GIS

Owner of MapWindow.nl - Support for
Dutch speaking users: www.mapwindow.nl

*******
Everything I say or write is my personal opinion and
not the opinion of the company I work for.
*******
View my profile on LinkedIn

Paul,

<<<If you want the mwlyr file and shapefile be related through the filename, the above scenarios wont work anymore and the mwlyr file has no use since it will work like the current lbl and mwsr files.

Why not? There can be several .mwlyr files with different indices, say states1.mwlyr, states2.mwlyr, etc. While loading shapefile it's quite possible to choose from which file the visualization options are to be loaded and to load the same layer with different options.

What do you think?

I'm rather indifferent to this, as it strikes me as a fairly esoteric feature. I would rather have lost functionality restored first so that I can actually use the new MapWindow for something.

(1) Since there was never any API for programmatically creating label specifications (or a new project file) or even a published spec for the .lbl (or .mwprj) file formats, plugin developers were forced to reverse engineer these formats and either create them in XML via New XmlDocument or else include blank files created in advance. But now MW ignores .lbl files.

(2) To force MW to label a newly added shape, the plugin could do this in its Message method:

MapWinIntf.Plugins.BroadcastMessage("LABEL_RELABEL " & Layer.Handle)

This would relabel without bringing up the labeler dialog. This is now broken, probably because the new labeler (?) no longer handles this message.

(3) To relabel a layer, a plugin could do this:

MapWinIntf.Plugins.BroadcastMessage("LABEL_EDIT " & Handle)

This is now broken. Or rather, it now brings up a dialog that no longer is able to relabel a layer.

I don't understand the splitting of label stuff into two dialogs. Most of the stuff in the new layer Properties dialog (which is confusingly title "Symbology" rather than the expected "Properties") means nothing to me (or to users), so picking out the bits that are relevant to labels is problematic. Please restore a dedicated label dialog that makes sense to users.

Thanks.

-Phil

Phil,

1) The support of .lbl format will be restored. Though I need to elaborate the prinicipal moments I described in the first point. So let's wait more comments before moving further.

2) LABEL_RELABEL message will be added. It's needed - no questions.

3) I'll think over restoring relabeling from the form (LABEL_EDIT). In case LABEL_RELABEL call won't be enough, then it's necessary indeed.

<<<I don't understand the splitting of label stuff into two dialogs.

The reason is quite clear - there is too many options for labels now. The window will be difficult to understand. Also old labeler infact did 2 different tasks:
- calculation of label positions (takes a lot of time and is performed usually only a single time for the layer);
- settings label properties (fast and is performed more often).
I personally felt quite confused with the old form, which did both tasks, which resulted in the unnecessary recalculation of label positions. When working with large shapefile I suppose we need to avoid such situations.

But if the problem exists - ok, I'll naturally try to do it more understandable. If nothing will help - it's possible to revert to the old form indeed.

Thanks,
Sergei



Edited 1 time(s). Last edit at 04/13/2011 09:32PM by Sergei.

Suggestion: labels

The principal thing I don't like about labels now is XML format for their storage. It's quite time prohibitive while loading/saving. In fact it becomes a bottleneck while working with really large shapefile (~10^6 shapes). Probably makes sense to consider the implmentation of the binary format which can be 10 times faster. It can work like this.
- binary file holds positions of the labels (X, Y, rotation, category);
- project file holds labeling expression, something like: [State_Name] + newline + [Area] + " ha"
- while loading the the project, first, empty labels are generated and, second, text is generated based upon expression.
In such a manner we avoid duplicating the text information which is present in dbf all the same and ensure that labels text was updated on loading. It's somewhat a question with speed of analysing the expression though. But It's possible dump text to the binary file as well (with som kind of index as record length will be variable). Of course such stuff, if implemented, must be embedded in ocx.

What do you think?

Thanks,
Sergei

Defining labels as an expression is a great idea. Currently I have two special fields in my shapefile's .dbf just for labels since there's no way currently to add, say, units to a field value. Originally MW had only one line label, but they added a second line at my suggestion. What you're suggesting is a much better approach and is similar to "calculated" fields in a database that don't take up any space and are made available "on demand" so the values are always up to date.

Question: If the attribute table is edited, either interactively or programmatically, will this trigger a relabeling or at least some way to do that?

If you limit label expressions to the concatenation operator (+) the expression parser should be very easy to write and very fast. However, if you allow, say, IIf() in the expression that would provide a way of generating a label conditionally and would be very powerful indeed. In my experience, simple expression parsers can be very fast even if they support a lot of functions and operators.

I'm not so sure that a binary format for storing label numeric values is a good idea, though. Binary formats have a habit of coming back later and biting you, either because of endian issues or file corruption or just the general inability of anyone else to work with them or repair them without the proper driver. That's the huge advantage of XML. Is XML really that slow once you've removed the label text from the label file and only generate label text on the fly?

Certainly I would be happy to re-do my plugin's labeling as long as there's a way to programmatically define labels and relabel them when needed.

Thanks.

-Phil

Phil,

actually I've implemented a expression parser last summer which is already used for :
- shapefile drawing categories;
- label categories;
- visibility expressions.

I also added Shapefile.Labels.Expression property and it parses the expression ok (it's not yet committed). However in current version no string or iif functions are supported. It's possible to add them indeed, but now I'm trying just to make everything work smooth. So functions - later.

But I can add TextExpression property to the LabelCategory class easily. It can work like this:

Category.Expression = "[Population] > 10^6"; // ^ operator is supported
Category.TextExpression = "[Name]" + Enviroment.NewLine + "[Population]/10^6 + mln."; // currently syntax is somewhat more difficult (\" are needed for string constants)

It can be a good substitution for IIf. What do you think?

As for binary format thanks for info. It needs thinking indeed. I suppose that just parsing X, Y values from XML will be still time consuming. But that's just guessing - tests are needed here.

>>>Question: If the attribute table is edited, either interactively or programmatically, will this trigger a relabeling or at least some way to do that?

It's possible to update Shapefile.Labels.Expression property even now (pass the same string once more):
Labels.Expression = Labels.Expression;
Additional method can be added as well, like Labels.UpdateText.

Triggering automatic relabeling is more complicated. Probably possible to set some bool variable in table class or for each row, say bool _valuesChanged; and to update the label text before the map redraw if it was set to true. But it will increase complexity of the code considerably, so I'm against it.

Thanks,
Sergei



Edited 1 time(s). Last edit at 04/14/2011 07:51PM by Sergei.

Instead of writing and parsing X and Y values as XML we could store label locations as a point shapefile. That is a familiar standard binary format we are already good at reading and writing. Imagine we save label locations as OriginalName.labels.shp by default.

I am not sure what to do about the label strings themselves and the various properties (font, color, rotation, etc.) that may be the same for the whole layer or might be different for each label.

A straightforward solution would be to store label strings and properties of each label in separate columns of the associated OriginalName.labels.dbf.

A separate shapefile for labels, if complete with a .dbf, would be easy to load in another GIS.

I am not sure whether this or regenerating labels from a pattern will be the best tradeoff between efficient disk space vs. quick/easy loading, rendering, or regenerating labels.

Label properties that are the same for all labels in a layer (all may be the same font or color for example) would be somewhat wasteful to include in columns in the .dbf but I am not sure where else to put them. Perhaps in the .mwlyr file? On the other hand, sometimes it will be desirable to store some properties on a per-feature basis (custom rotation of each label to fit inside its polygon) and .labels.dbf would be a natural place for that.

We could add the labels layer as a separate item in the legend with its own properties or we could try to hide its existence within Label properties of the original layer. Keeping the labels synchronized with changes in the original layer sounds like it might be a bit of a challenge either way.

Mark Gray Wrote:
-------------------------------------------------------
> Instead of writing and parsing X and Y values as
> XML we could store label locations as a point
> shapefile. That is a familiar standard binary
> format we are already good at reading and writing.
> Imagine we save label locations as
> OriginalName.labels.shp by default.

Another great idea and a good way to recycle existing technology.

Couldn't Sergei's idea of an expression to define a shapefile's labels still be used? Perhaps I'm not understanding his proposal, but wouldn't whatever text the expression resolves to for a given shape be stored in the .labels.dbf, for example in the TEXT column (or whatever it's called)?

Questions: Is there linkage between the original shapefile and its label shapefile? For example, if a shape is deleted or added via the shapefile editor or programmatically, what happens to the .labels.shp?

Thanks.

-Phil

If we want to, we can work on the new format to make it backward compatible and use the .mwsr extension. We would need to do some testing of how robust MW 4.6 and 4.7 are when reading "new improved" .mwsr files. I imagine it may not be easy to extend the old format to include all the new information without either making older MW unhappy or making the format hard to read because it mixes old and new keywords. I think using the new file format and the new extension .mwlyr for it is a fine choice that may require less effort than extending .mwsr.

Thinking ahead, we should make our reading of the .mwlyr very robust so we can still read the parts of the file we know about today even if in the future a newer MW writes a new .mwlyr that has new parts or leaves out an old part of the file. MW used to be bad about this with its .mwprj and I spent some effort making it keep going even if some small detail of the XML was missing.

For backward compatibility, we still want to be able to read the older .mwsr, .mwleg, .lbl files, but we do not need to write them any more if we have a new replacement and we only need to look for and read the older files if there is no .mwlyr that we can tell is associated with the layer.

Opening .mwlyr from OpenFileDialog seems quite desirable if there might be more than one for a file or if they may not even refer to a file but instead a web service.

Storing more than one .mwlyr file for the same layer sounds like a useful concept, though I can see how this may be confusing to users. We should try to make the difference between files clear, perhaps by having the user name the .mwlyr file as it is saved. Maybe the convention is like LayerName.RendererName.mwlyr.

Maybe load and save of .mwlyr are supported in the layer properties dialog along with naming the .mwlyr instead of saving as a separate legend menu item? Not saving would mean the information is only saved inside the .mwprj. Saving but not giving it a name could just save it as LayerName.mwlyr.

I see it as desirable to try to automatically find a .mwlyr that describes a way to show a layer when a .shp file is added to the map, but that does not mean we can't have more than one .mwlyr.
There are a couple of strategies for how to search for a .mwlyr in that case. We could look at only the files matching LayerName*.mwlyr or we could look inside all *.mwlyr for any that refer to this .shp.
If we find none, we look for the older files like .mwsr.
If we just find one, we use it and do not prompt the user.
If we find more than one .mwlyr, we could just use the first or ask the user which to use.

If we do prompt the user for which to use, the idea of previewing each choice is interesting. I am not thinking it will be common for users to have more than one .mwlyr for the same layer and when they do they can just add the one they want instead of the layer file itself, so I'm not sure whether it is worth the time to
develop the preview-different-.mwlyr dialog, but it would be cool.

Mark Gray Wrote:
-------------------------------------------------------
> For backward compatibility, we still want to be
> able to read the older .mwsr, .mwleg, .lbl files,
> but we do not need to write them any more if we
> have a new replacement and we only need to look
> for and read the older files if there is no .mwlyr
> that we can tell is associated with the layer.

Yes, this sounds very reasonable.


> Opening .mwlyr from OpenFileDialog seems quite
> desirable if there might be more than one for a
> file or if they may not even refer to a file but
> instead a web service.
>
> Storing more than one .mwlyr file for the same
> layer sounds like a useful concept, though I can
> see how this may be confusing to users. We should
> try to make the difference between files clear,
> perhaps by having the user name the .mwlyr file as
> it is saved. Maybe the convention is like
> LayerName.RendererName.mwlyr.
>
> Maybe load and save of .mwlyr are supported in the
> layer properties dialog along with naming the
> .mwlyr instead of saving as a separate legend menu
> item? Not saving would mean the information is
> only saved inside the .mwprj. Saving but not
> giving it a name could just save it as
> LayerName.mwlyr.

Something about sounds like a usability nightmare. Opening the .mwlyr separately? You mean from File or View menus? That's just sounds like bad UI design.

Is there some reason why multiple layer definitions can't be stored in the same .mwlyr file? Why do we need multiple .mwlyr files?


>
> I see it as desirable to try to automatically find
> a .mwlyr that describes a way to show a layer when
> a .shp file is added to the map, but that does not
> mean we can't have more than one .mwlyr.
> There are a couple of strategies for how to search
> for a .mwlyr in that case. We could look at only
> the files matching LayerName*.mwlyr or we could
> look inside all *.mwlyr for any that refer to this
> .shp.
> If we find none, we look for the older files like
> .mwsr.
> If we just find one, we use it and do not prompt
> the user.
> If we find more than one .mwlyr, we could just use
> the first or ask the user which to use.
>
> If we do prompt the user for which to use, the
> idea of previewing each choice is interesting. I
> am not thinking it will be common for users to
> have more than one .mwlyr for the same layer and
> when they do they can just add the one they want
> instead of the layer file itself, so I'm not sure
> whether it is worth the time to
> develop the preview-different-.mwlyr dialog, but
> it would be cool.

I think it's important not to lose sight of the prize. While I'm sure there's a certain amount of MW ad hoc use by GIS techies and what not, as programmers I think we all understand that the real power of MW is in highly specialized plugins that automate and idiot-proof the repetitive things and guide the user. My users are not "GIS" people. I need to create a turnkey solution. All this talk of prompting the user and loading separate .mwlyr files makes me a bit nervous.



> MW used to be bad about this with its .mwprj and I
> spent some effort making it keep going even if
> some small detail of the XML was missing.

In the past, it wasn't possible to store any extra data at all in the .mwprj file because MW would just write over this data the next time it saved the project file. This always struck me as a waste of using XML, which specifically supports "two-way" tools that can handle data that they don't know about. Has this changed any? My current solution is to create my own separate "project" file (different extension) in order to store the small number of additional bits of data that should really be in the master project file.

Thanks.

-Phil

ABOUT LABELS:

<<<I am not sure what to do about ... various properties ... that may be ... different for each label.

Just for clarification. There is not such thing as per-shape label properties in the new implementation of labels. There are label categories instead. A category define a set of options. For labels that belong to the given category, Label.Category property is set (equal to category index).

This approach is more flexible, ensures faster drawing and decreases RAM usage.
Of course in the worst case scenario, when properties for all labels are different (number of label categories is equal to number of labels), the method will be slower. But in reality it's difficult to imagine a situation when more then a hundered categories is needed (different sets of options), while the number of labels can be much much greater.

It's quite natural to store LabelCategory options in the XML (.mwlyr or .mwprj). Usually they won't take too much space, especially when storing only the values different from default ones.

<<<Instead of writing and parsing X and Y values as XML we could store label locations as a point shapefile

It sounds interesting, but upon consideration I think it'll provide quite a headache to both developers and users. Though the idea of exporting labels as a separate point shapefile sounds good to me (as a mean to communicate with other GIS - could be added to the GIS Tools I suppose).

I've just checked how QGis handles labels. It stores neither coordinates no text in the external files, but uses fields of the shapefile itself to do it. In our case it can be enough to set:
X, Y, Rotation, Category.

The disadvantage - multipart shapes can't be labeled properly in such scheme (the largest part only). But it's quite possible to run Explode on the shapefile to fix it.

The main advantage - simplicity. So from the first glance - I'm for it. Are there any disadvantages I missed?



Edited 1 time(s). Last edit at 04/15/2011 05:06PM by Sergei.

ABOUT RENDERING OPTIONS:

I largerly agree with Phil's comments. Whatever solution we choose it should be simple enough. It's just a matter of good UI design and saving the nerves. Also the idea of storing all the definitions in one .mwlyr file looks good to me.

So here is the approach I see:

1. There can be a number of rendering settings for the layer, but they all should be kept in a single .mwlyr file. The only problem here is potentially excessive size. But if we abstain from saving per-shape data for labels and charts it won't be a problem. There are also textures, but their size can be limited.

2. One of the option sets in the .mwlyr file should be marked as default one, the existence of a single default set should be maintainted programmatically.

3. Old formats (.mwsr, .lbl, .mwleg) should be supported on the equal basis with .mwlyr option sets.

4. No opening of the .mwlyr file (or older files) from OpenFileDialog should be allowed. For web services when they appear, another extension can be used.

5. When opening a layer with multiple rendering options a project setting should be used, say 'Rendering info loading type' with following options:

a) Load with random options
b) Load with default options
c) Prompt the user to choose options

Default options (b) should work like this: if .mwlyr exists, then there is only one set of options is marked as default and we choose it, if .mwlyr doesn't exist but old files do - they will be treated as default options). If none of the files exists random options are used.

User prompting (c) should work like this: if either of the files exists a prompt will be shown to choose from:
- random options;
- all available sets of options including old ones;
If none of the files exists random options are used.

I suppose by such approach we can satisfy both beginners (deafult options) and advanced users (user prompting).

6. I'd still like to make preview for user prompting. When shapefile is relatively small it can be updated on the fly, if large - by additional 'Update preview' button. Annotation for each option should be available.

7. A form should be added to manage the sets of options (save, delete, annotate, choose default), either as Layer Properties tab or as a separate window. Preview capability is very much desirable for it as well. A fast saving of current options should be preserved also (from pop-up menu of the layer - like it's now). Options saved in such a way should be made default automatically.

8. Some way to get rid of the old files (.mwsr, .mwleg, .lbl) should be deviced. For example, if old options where loaded, automatically write new .mwlyr file (by serializing the layer) and after checking that it was actually written - delete all the old files.

9. Fully agree with Mark on the robustness issue of the .mwlyr. I can add that it's very desirable for this file to be consumable by MapWinGIS directly. Probably with the index as parameter for the desired set of options (-1 for the one marked as default).

What do you think about such approach?

P.S. After reading Phil's comments on the newbies. Probably makes sense to add project setting 'Beginner/Advanced' user and not to show certain tabs from the layer properties window to the beginners. Some options there are indeed not easy to understand and intended for advanced users only (like Mode tab or Visibility expressions tab). Maybe no need to racket their brains ;)



Edited 1 time(s). Last edit at 04/15/2011 06:57PM by Sergei.

This has become a good discussion!

Reading Sergei's last posts I think he found a good consensus.
I don't like deleting old files (.mwsr, .mwleg, .lbl), though. Perhaps we should make a back-up first by zipping them into a file called layername.oldRender.zip Then the files can be deleted.

I also like the idea of saving the label settings in the dbf-file. If the field names would start with '_' we can hide them from the table editor.
Using the Spatial Converter it will be easy enough to convert this dbf file to a point shapefile if necessary.

I'm still puzzled how to handle my initial request: Opening a shapefile twice with different render options, without copying the whole dataset.
If MW doesn't support opening a .mwlyr file as a layer my initial request won't work, right?
But perhaps my request is to exotic and will it never be used. We could also solve this by adding an option to the context menu in the legend to copy this layer (or save as ..).

It would be nice if the ocx could read and write .mwlyr files and .mwproj files, but I don't know if it is feasible to do this in this version. A lot of work is still to be done and we're working on v4.8 for more than a year now.
We really need to finish this version and release it during the MapWindow conference in San Diego. So I want to suggest to move this 'creation in the ocx' to v4.9. Of course this can still be the first thing to implement.

I would like to ask Mark en Phil to reply if Sergei's approach is workable or not, so he can continue.

Thanks,

Paul

--
Don't forget to read the new documentation: www.mapwindow.org/documentation/mapwingis4.8
Join us Google+: MapWindow GIS Google+ Community
Join the MapWindow Group on LinkedIn! LinkedIn - MapWindow Group

Download the latest beta installer at:
tinyurl.com/mwMonthly 32-Bit
tinyurl.com/mwMonthlyx64 64-Bit
Follow me on Twitter MapWindow_nl to read when a new installer is published.

---
Paul Meems
The Netherlands
[www.bontepaarden.nl]
Release manager, configuration manager and
forum moderator of MapWindow GIS

Owner of MapWindow.nl - Support for
Dutch speaking users: www.mapwindow.nl

*******
Everything I say or write is my personal opinion and
not the opinion of the company I work for.
*******
View my profile on LinkedIn

Paul and all others,
YES, this has become a good discussion.
BUT, I feel a little left out because I do miss a definite list of all file types used by MW with explanation of what they are good for.
From the discussion I can deduce some of the purposes of some of the different file types. However, there is no concrete description of what they contain, which parameter is current and which is carried forward for backward compatibility.
Don't those of you in the know feel embarrassed about this lack of information?
Cheers, Gerd

Sergei Wrote:
-------------------------------------------------------
> ABOUT LABELS:
>
> I've just checked how QGis handles labels. It
> stores neither coordinates no text in the external
> files, but uses fields of the shapefile itself to
> do it. In our case it can be enough to set:
> X, Y, Rotation, Category.
>
> The disadvantage - multipart shapes can't be
> labeled properly in such scheme (the largest part
> only). But it's quite possible to run Explode on
> the shapefile to fix it.
>
> The main advantage - simplicity. So from the first
> glance - I'm for it. Are there any disadvantages I
> missed?

On the face of it, storing label information in the attribute .dbf sounds like a very bad idea. This is confounding the view model with the data model. Attribute data should be only that: data that describe what the shape represents in the real world. Label information and things like shape color, line thickness, etc. is how the user want to view those shapes.

I'm surprised that QGIS does this. They're cross-platform, meaning they should have been exposed by now to the MVC architecture that Apple _strongly_ encourages.

I feel like we're drifting back to the assumption that MW will be used primarily in an ad hoc way by GIS boffins. I don't think that is (or should be) its primarily use case. Certainly it's not as good of an ad hoc "viewer" as QGIS is. However, MW is a much better place to develop specialized plugins for several reasons, of which two are (1) can use any .NET language instead of Python or C++, (2) avoids QGIS's toxic GPL license.

Maybe we need a concrete example to discuss. Here's the example MW project included with our plugin:

[ftp.agry.purdue.edu]

Here's our plugin:

[ftp.agry.purdue.edu]

Here's an installer that installs both if you'd rather do it that way - just uncheck the MapWindow CF component to use the plugin and example project with your own MW.

[ftp.agry.purdue.edu]

Here's additional information:

[www.agry.purdue.edu]


The example project includes 17 shapefiles. All of the .lbl files and most of the shapfiles were created by our plugin, not by the user. Also, many of the shapefile .dbf's already include label fields, forced there because MW had no way to do labels via an expression or on the fly.

I would suggest this course of action:

(1) Restore support for existing .lbl files and labeling messages.

(2) In designing the new labeling system, please advise us on how we should proceed. I'm open to most anything. For example, should we continue creating .lbl files as now? Should we create new .mwlyr files for new projects and for new layers in existing projects? Should the plugin assist in converting to a new label system (e.g., by intercepting the mnuOpen menu item and converting its own .lbl files)?

That sort of thing.

Thanks.

-Phil

Phil,

<<This is confounding the view model with the data model.

Do we understand it the same? It's needed actually to store 3 fields for labels:
- X;
- Y;
- LabelAngle,

These attributes actually easily fall to 'caching' definition. I wouldn't save them at all if they were fast to calculate. But they are not. So it's possible to consider them as additional description of data (the models for data can be different, so why not to hold centroid, for example, in pre-calculated state if we use it often?).

User preferences, like color, line thickness are and will be stored in the external file (.mwlyr).

Labels will be expression-based, so no text strings will be stored separately.

What do you think?

Thanks,
Sergei

That sounds good. Just don't store X, Y, LabelAngle in the .dbf - I don't believe they belong there. Also, I don't think it's a good idea for MW to just add its own fields to a .dbf, particularly for .dbf's that originate elsewhere and may be used elsewhere (and are essentially used in a "read-only" capacity - only view the data, not change it).

Thanks.

-Phil

It sounds like we have four options on the table right now for storing label locations, rotation, and category: .mwprj, .lbl, add to .dbf, and create separate .shp.

After reviewing all of these, it makes the most sense to us at Aqua Terra to simply return to using the .lbl file as MW4.7 did, using the same format and tags as before to maintain backward compatibility but adding any new properties that were not available in MW4.7.

---------

.mwprj is where MW4.8 is currently putting all of this information. This uses very similar XML to the .lbl file and has most of the same advantages and disadvantages as .lbl listed below except:
it adds label categories as a new feature
fewer files are written because everything is included in the .mwprj
it is not backward-compatible with MW4.7 and existing projects and plugins

---------

.lbl is an XML file written alongside each labeled shapefile.

Advantages:
backward compatible with MW4.7 and earlier
existing users already have .lbl files, so we should at least support reading
some existing plugins already create .lbl
DotSpatial can read .lbl and draw labels from it
does not require any changes to the original .shp or .dbf
provides a place to put locations, rotation and label text for each label
provides a place to put rendering properties for the entire layer

Disadvantages:
XML is slower to parse than binary or .dbf (only important for large shapefiles)
contains a duplicate copy of label information that already appears in .dbf
not automatically kept in sync with changes in the shapefile
not yet understood by MW4.8 or any other GIS.
additional file clutters the folder and can become separated from its shapefile
rendering properties are for the entire layer, no way to vary by category
(but we could probably add a category attribute to each label)

---------

add to .dbf means adding columns to the shapefile's existing .dbf. This would require choosing a column format for X, Y that will work for all scales.

Advantages:
automatically deletes labels of shapes deleted from shapefile
(but like others does not automatically add/change label information for added or moved shapes)
faster to parse than XML
many programs already know how to open dbf
can add field for label category to vary label rendering within a layer
does not add to the number of files in the folder

Disadvantages:
adding fields to dbf changes user's data file
numbers in dbf are slower to parse than binary (all dbf fields store as text)
X, Y in dbf not yet understood as coordinates by MW or any other GIS.
rendering properties are for the entire layer, could add another field to vary by category

---------

create separate .shp means making a new point shapefile to store label locations. A PointM format file would have room for X, Y and Rotation in the shp. Could store category and label text in fields of a new dbf

Advantages:
fastest reading of location and rotation fields
location information is already understood by all GIS (but not rotation, category)
does not require any changes to the original .shp or .dbf

Disadvantages:
creates at least one and probably three additional files (shp shx dbf)
may contain a duplicate copy of label information from original dbf
not automatically kept in sync with changes in the shapefile