So I’ve been meaning to write a post on this ever since I had a great talk with Andrew Turner, who recently joined Esri. He was expressing a good bit of optimism over Esri’s willingness to embrace ‘open’. My worry is that they would embrace the language of open, but not actually do the key things that would make a real open ecosystem. It’s easy to take enough action to convince more naive decision makers of their intentions, like esri.github.io, which looks like lots of open source projects, but is mostly code samples and api examples that are useless without pieces of closed software. So I wanted to give to Esri a measurable roadmap of actions to take that would signal to me a real commitment to ‘open’.
That was a couple months ago, but then life got in the way, as tends to happen with lots of planned blog posts. But in the meantime there’s been a fascinating flare-up around the GeoServices REST specification in the OGC (most conversation is behind OGC walls, but see open letters and lengthy discussion on OSGeo). And it similarly makes me want to address ‘Esri’ as an entity. The individuals matter less, I believe that most of them want to do the right thing. But the corporation as a whole takes action in the world, and that big picture matters more than what individuals within are trying to do. So here goes:Dear Esri, I imagine you’ve been pretty confused and flummoxed by all the push back to the GeoServices REST specification. It is a pretty solid piece of technology, and theoretically is a great step towards opening up Esri more – putting previously proprietary interfaces in to an open standard. The key thing to understand though is that unfortunately very few people in this industry trust you. I am actually fairly unique in that I give you a whole lot more benefit of the doubt than most. In my life I try as hard as I can to only judge people based on their direct interactions with me, not on whatever people say about them. And though you aren’t exactly a person I do still try to judge in the same way. And I personally have never had a truly bad interaction with you. But amazingly just about everyone that I’ve encountered in the broader geospatial industry has. When I bring up that I’ve never been mistreated by you they have the utmost confidence that I inevitably will. I have always been fascinated by this, as I believe it goes far beyond ‘typical competition’, which is one thing people at Esri have told me – something like ‘they’re just jealous’. The amount of venom that otherwise totally decent people will throw out is incredible, and it makes it hard for me to not judge harshly, when so many people whose judgement I trust absolutely have felt really screwed in the past by you. Multiple times. In really unpleasant and unexpected ways. And it’s not just competitors, but partners and clients. So the fundamental point that needs making with regards to GeoServices REST was articulated well by one of my allies: ‘everyone in the OGC membership is objecting to the REST API because they believe ESRI to be fundamentally conniving and untrustworthy.’ He believes that if the REST interface had been proposed by anyone else then there wouldn’t be a problem – it would likely be an approved OGC standard by now. But people believe it’s a ‘cynical business play by an untrustworthy, well resourced, and predatory business interest.’ I have overall had a relatively positive take on the GeoServices REST API, because I don’t (yet?) believe you to be fundamentally conniving and untrustworthy. You’ve done an amazing amount good for this industry, but I do think you’ve just lost your way a bit. I imagine you think that putting this great interface in to OGC is a great step in the right direction, and it is, but unfortunately it’s too much too fast. It’s as if the schoolyard bully is suddenly super nice to you – you wonder what’s up his sleeve, how he’s going to screw you this time. But I believe there’s a path forward, to building up people’s trust so that something like GeoServices REST could be accepted in OGC. It just has to be slower and more incremental. This list is just what I can think of right now. There is a fundamental principle at work though, which is moving towards an architecture that encourages people to mix and match Esri components with other technology. Not merely implementing open standards to check the box, but building for hybrid environments: QGIS exporting to or editing ArcGIS Server, ArcGIS Desktop doing the same to GeoServer, TileMill styling a file geodatabase and publishing to ArcGIS Online, ArcGIS Desktop styling and publishing to Google Maps Engine or CartoDB, etc, etc. Each piece of Esri technology ideally could be used stand alone with other pieces. Stated another way, there should be no lock-in of anything that users create – even their cartography rules. Anything that is created with Esri software should be able to be liberated without extreme pain, in line with the Data Liberation Front (though I think the google maps engine and earth enterprise teams also need som help from them, they too deserve a similar letter). I realize this is a big leap, since it is not the absolutely most efficient way to solve the needs of most of your customers, since most of them use only your software. And it is a business risk, since it opens up more potential competition. But it’s also a big business opportunity if done right. And reaches beyond mere business to being a real force for good in the world, becoming a truly loved company, with lots of friends. This list is roughly ordered by easier wins first, and so later ones should probably build on them. If these actions are taken I will start to defend you to my friends a lot more.
- Enable W*S services by default in ArcGIS Server. You’ve done a pretty great job of doing CITE certified implementations. But as the docs show they are not enabled by default, though GeoServices REST and KML are.
- Make GeoJSON an output option for the ArcGIS Server REST implementation (and get the clients all reading it). And then get it in the next GeoServices REST specification version.
- Publish the file formats of file geodatabase. The open API was a really great step, and indeed goes 80% of the way. But many libraries would like to not have to depend on it. We all know the format itself is likely an embarrassing mess, but everyone will forgive, as we’ve all written embarrassing code and formats.
- Help the coming GeoPackage be the next generation of file geodatabases. Help it evolve to be the main way Esri software moves data between one another. Your team has been great on it so far, but the key step is to make it a top level format not just in mobile but also on ArcGIS Desktop (reading and writing it, like a shapefile), Server and Online (as an output format and upload configuration format for both). And I’m hoping your team can join our plug-fest this week, to get to 3 real implementations before version 1.0 of the specification.
- Stop the marketing posts about how open you are. Open is action and conversation, not a press release or a case study. No need to talk about how we’ll be ‘surprised’ by what you open in the future. Just start opening a lot and let others do the posts for you, and count on that reaching your customers if you’re doing it well.
- Openly publish the .lyr file format (and any other formats that define cartography and internal data structures for maps). This is probably an internal format that isn’t pretty, but opening it would enable a lot of cool hybrid architectures.
- Support CartoCSS as an alternative to .lyr file, and collaborate around advancing it as an open standard to ideally make it the next generation .lyr file. This likely will include a number of vendor extensions as your rendering capabilities are beyond most everyone else. But we can all collaborate on a core. Supporting SLD in desktop would also be great, but we can also just look to the future.
- Move WFS and GML support out of ‘data interoperability‘ extension and in to core format support
- Bring support for WFS-T to ArcGIS Desktop, so people can use their desktop to directly edit a WFS-T server. The server side support in ArcGIS Server is great, but the client software also needs to implement the open standards for real interoperability. I think a similar thing might be needed for Desktop to edit directly with GeoServices REST, though maybe that is there now.
- Become a true open source citizen. This is another large topic, and as is my tendency I’m already going on too long. So perhaps I will detail it more in another post. I have limited the above to mostly standards, but embracing open source can take you even further. But it is much more about contributing to existing open source libraries and building a real community of outside contributors on your projects, not just releasing lots of code on github. Karl Fogel wrote an excellent book on the subject. You are making some progress on this, but it is just a smidgen if you are actually serious about opening up. I will give props for the flex api, though my cynical friends would say it’s the easiest one to open source as it is dying. So please, do continue to surprise us on the open source front, you just don’t need to talk about it a bunch.
(Written as a private citizen, who cares about our collective future and believes geospatial has a higher role to play than just an ‘industry’. Views expressed here do not reflect my employer or any other organizations I am associated with)