Don’t Copy Stuff From The Library

Today I had to ask a scenery author to withdraw their scenery from the Download Manager at x-plane.org because he had copied a large number of objects out of the library into his scenery package.

This is a call to all authors:

Please download the Developer Pack from the home page and read the instructions inside it – you do not need to copy any files to be able to use OpenSceneryX.  If you do, you are breaking the license and infringing the copyright of the generous people who have contributed to the library.

This has happened on a few occasions, always due to innocent mistakes and usually just one or two objects.  It’s easily fixed but it’s better if it doesn’t happen at all.  Please read the documentation in the developer pack – it’s very easy to use the library properly.

Old Liveries

Airlines change their liveries from time to time, but why should this bother us?  As I mentioned in a previous post, some collections of aircraft are published to shared paths and some are not.  If an author uses a path that several aircraft are published to then X-Plane will pick a random aircraft each time the scenery is loaded.  For example, most b747-400s are published to /objects/aircraft/jets/heavy/b747-400.obj but some one-off liveries (Air Force One, for example) are not included in this, so you will never get a randomly placed Air Force One – the scenery designer has to explicitly choose to place that aircraft for it to appear.

One specific category of aircraft that is excluded from random placement are those aircraft that are in historical liveries. Historical liveries are present in the library because some authors may want to use them under specific circumstances, however when randomising we don’t really want old liveries appearing, as it would look strange in a modern airport.

So when does a livery become historical?  Well, you may say, surely it becomes historical when the airline starts painting a new one on its aircraft?  Well, I say, but what if no-one has contributed the new livery to the library yet?  Should all British Airways aircraft simply stop being placed randomly the moment BA change their colours, until some generous person contributes a new livery?

The answer is no.  The old livery remains active and current until a new one is submitted.  At that point the old livery is moved into a sub-path (e.g. /objects/aircraft/jets/heavy/b747-400/british_airways.obj would move to /objects/aircraft/jets/heavy/b747-400/british_airways/historical/1984-1997.obj and the new livery would replace the old aircraft).

Virtual Paths

If you’re a scenery developer, and you open up OverlayEditor and see the list of objects in OpenSceneryX, you are looking at a set of Virtual Paths to the objects.  These are not real paths on disk, they are made-up.  Hopefully I’ve done a reasonable job of making up these paths so that it is easy to find the object you are looking for.  Sometimes this goes a bit wrong and I have to change things around a bit, see the previous post for an explanation of how this is done.

Virtual paths give us three benefits:

  1. The folder structure on disk does not have to bear any resemblance to the path that an object is published to.
  2. An object can be published to more than one (indeed, many) paths.  However, it is still the same object and is only loaded once by X-Plane.
  3. Several different objects can be published to the same path.

Item 1 is good for library maintainers – We can organise the files however we like, as long as the published paths stay the same as time goes by it doesn’t matter where things sit on disk.

Item 2 is good for everyone – This allows objects to be found in different places, so they can be organised under different schemes and found by the user more easily.  A good example are the static aircraft in the library – These are organised both by aircraft type and by livery…  So if you are looking for a B747 then you can find it under /objects/aircraft/jets/heavy/b747-400/aer_lingus.obj but if you are looking for Aer Lingus aircraft you will find the same 747 here: /objects/aircraft/livery/aer_lingus/b747-400.obj.

Item 3 is also good for everyone – Publishing multiple objects to the same virtual path means X-Plane can introduce random variety as the sim will pick a random object each time the scenery is loaded.  A massive example of this is /objects/aircraft.obj – Almost every aircraft in the library is published to this path, so if you use it you will get a random aircraft each time – a bit extreme!  Perhaps more useful are paths like /objects/aircraft/jets/heavy.obj (a random heavy jet) or /objects/aircraft/jets/heavy/b747-400.obj (a random b747-400).  Note that some aircraft are excluded from this, specifically historical liveries, house colours and one-off liveries.

I’ll blog a bit more about historical liveries in future, as well as blogging about the build scripts that are used to compile the library.

Backward Compatibility

Metal Barrier
Metal Barrier

Ok, so here’s the problem: When an object is first contributed a decision is made to publish it to a particular path (e.g. /objects/furniture/barriers/portable), the library is built and the object is published to everyone in the next release. Time passes and the library grows and authors start using the object…..

…after a while and a few more releases it becomes obvious that the original path the object was published to is not really very good… usually because a bunch of other objects get contributed and a better way of organising them springs to mind.  A decision is made to move the object (e.g. to /objects/furniture/barriers/metal/1).  The problem is that older scenery packages that reference the object at the previous path would break because they can no longer find it.  To solve this, we have the concept of deprecated paths in the library – take a look at the docs for the above object – it shows that the object used to be published at one path, but from version 1.8.5 it is also published to another.

This makes newer versions of the library backward compatible with older ones and hopefully means that scenery package builders can always rely on their packages working with future releases of OpenSceneryX.

Um… That is until the library maintainer makes a mistake and forgets to add in a deprecated path when an object has moved, which happened recently with /objects/airport/vehicles/stairs which moved to /objects/airport/vehicles/stairs/1. Because there are so many objects in the library now, it is unlikely that I will spot these on my own, so if you discover a problem like this, please let me know straight away and I will fix it in the next release.  Backward compatibility is a very important goal and errors like this must be fixed immediately.

I’ll blog again shortly with more technical detail on how X-Plane allows us to do all this.

Important Info For Scenery Developers

IMPORTANT FOR ALL SCENERY DEVELOPERS USING OPENSCENERYX: Please read the instructions in the developer’s pack.  It is especially important to read the part about including the “Placeholder Library” in your package.  If you don’t do this, your scenery package may* CAUSE X-PLANE® TO QUIT if a user hasn’t installed OpenSceneryX.

I’ve noticed that a number of released packages don’t include the Placeholder Library and have tried to alert the authors whenever I spot this.  If you don’t do it you’re simply making work for yourself because you’ll get a bunch of users complaining that “I installed your package and it crashes X-Plane”

It is simplicity itself to include the Placeholder Library in your package, it’s just a file and a folder that you copy across.

For more detailed info, download the latest developer’s pack, which is always available from the OpenSceneryX Home Page.

(*) Versions of X-Plane® older than 9.30 beta 8 will quit, versions of X-Plane® newer than that will throw up a warning message.

The OpenSceneryX Incremental Installer

A quick exposé on how the installer works:

  1. The installer firstly checks whether it needs to update itself and will tell the user to download a new version if one is available.
  2. Next it scans the user’s existing local OpenSceneryX folder (if one exists), gathering information about every file it finds and creating a CRC checksum for each.
  3. It then downloads a manifest from the OpenSceneryX website.  This is an XML file that contains an entry for every file that exists on the server, together with its file size and a CRC checksum.
  4. Next, each entry in the manifest is compared to the information from the local file system and a list of new, changed and deleted files is generated.
  5. The installer then deletes all the local files that it no longer needs and downloads all the files that are new or that have changed.  It uses a standard HTTP (web) connection on port 80 to avoid firewall issues and every file that it downloads is individually compressed to reduce bandwidth and download time.

The basic principle will stay the same, but the installer is currently being revamped to be more user-friendly and will present a standard step-by-step install process, describing what it is doing at each step. Not sure when this will be out, but “soon” is probably quite a good estimate.

A General-purpose Scenery Package Installer?

There has been some discussion on x-plane.org recently about producing a more general-purpose installer of X-Plane addons.  This dovetails with something I have been considering for OpenSceneryX – The ability to select from a choice of OSX components, so a user can tailor their install to their own preferred set of content.

For example, OpenSceneryX could initially be split up into ‘Airport Buildings’, ‘Static Aircraft’, ‘Forests’ and ‘Other’ and the user could select which sections to download.

If this idea was extended to allow the user to pick scenery packages not just from OSX but also from other developers, then hey presto you have a global package installer.

As always, there would be a few hurdles to cross:

  1. There would need to be a central repository of all the sites where packages are available, and this would need to be kept up to date by each individual scenery author as they release new packages.
  2. A decision would need to be made whether this central repository was also where all the packages were stored, or whether each individual scenery author would be responsible for hosting their own packages on their own servers.  Of course, a central repository would need to be hosted and the usage could get quite high.
  3. Because an incremental installer is the best way to go (i.e. an installer that only downloads files that have changed since the last time the user checked) then each author would have to carefully process their content.  If they were hosting their own then this processing would need to be done by them but if it was all centrally hosted then the system could do most of it automatically.

This is basically all food for thought. I’ll post in future with more detail on how the OpenSceneryX incremental installer works.