[Open-transport] Open Transport Data tools / webservices

Tue Jul 16 05:42:16 UTC 2013

"some agencies are complaining that Google is not listening enough what
agencies want/need"

This is most certainly true.  At the end of the day, Google builds transit
features for riders first and foremost, and agencies don't always agree
with the decisions we make.  If you are a transit app developer, chances
are you also don't always agree with the decisions your transit agency
makes either and thought you might do better.  Of course, we all want to
help agencies where we can, but you have to acknowledge that there is
sometimes a fundamental tension there.

GTFS reflects that tension, as it was developed as a format for
communicating rider information (as opposed to operational information).
 There are certainly opportunities to improve exchange of operational data
(proposals to extend GTFS with a GTFS-operational spec, other formats, etc)
but part of what made GTFS successful (and relatively concise compared to
other standards) is its focus on doing one thing: communicating rider
information.  Replacing GTFS with something else might make some sense for
agencies, but I'm not sure riders will necessarily be better off.

Or perhaps you were just arguing that Google works with so many agencies
that it can't possibly provide responsive service to all of them?  That may
be true as well, but replacing GTFS with some other data format probably
won't fix that problem ; )

"GTFS is not a true consensus based standard"

I'd be interested in hearing more of your thoughts on this.  I certainly
try hard to make sure that GTFS is extended through general consensus, but
I also acknowledge that Google has an outsized influence on the process
given my activity on the GTFS-changes mailing list.  However, to me,
extending GTFS is hard not because any one participant is dragging their
feet.  It just takes a lot of work to see a proposal through from start to
finish, given the constraints defined in the GTFS extension guidelines.

Brian

On Tue, Jul 16, 2013 at 3:07 AM, Stéphane Guidoin <stephane at opennorth.ca>wrote:

> Brian, you are not easy to follow ;)
>
> Some thoughts about the previous points raised.
>
> Transit data have an advantage which becomes an issue: the data of each
> agency is pretty self-sufficient so that they don't have to bother about
> what the others are doing. I'm currently working on a traffic API (the
> traffic side of the open511 listed by Brian) and agencies have a higher
> incentive to use something common because agencies want to communicate
> together (for example state level agencies want to know what's happening on
> the local network of large cities and vice versa). (But event there, it is
> difficult to have them moving)
>
> The other issue here, is that we already have an elephant in the room:
> GTFS. As said by Brian, you will have difficulties to justify a new format
> given how long it has been to convince some agencies to adopt GFTS.
>
> The only lever that I see is the fact that some agencies are complaining
> that Google is not listening enough what agencies want/need (and that GTFS
> is not a true consensus based standard), which could become an incentive
> for some to quit, or not to adopt.
>
> I'm also wondering if/how something like the incoming-DAT (
> https://github.com/maxogden/dat) could be used for this need. Step 1:
> define a good API format. Step 2: list existing format that are good enough
> to comply with the good format Step 3: DAT/ETL.
>
> There are already some converters here and there, so it would be a
> question of grouping them together.
>
>
>
> On Mon, Jul 15, 2013 at 12:31 PM, Brian Ferris <bdferris at google.com>wrote:
>
>> tl;dr - Advocate for bulk data formats, not high-level service APIs.
>>
>> Creating service APIs for transit data is the easy part.  In fact, I'm
>> pretty sure we could find 20-30 different APIs that people have put
>> together if we tried hard (my contributions below).  Don't like the
>> existing APIs?  Create a new one!  http://xkcd.com/927/
>>
>> The real trick is getting everyone to use one API.
>>
>> At one point, I used to think it'd be great if every transit agency
>> provided a high-level REST API that I could point my app at and suddenly my
>> app would work in every city in the world.  However, when I started to
>> think about the technical details of making this work in practice, I
>> quickly decided it wasn't worth the effort.  In a perfect world, of course,
>> every agency would be running the latest version of the API and API
>> endpoints could be nicely federated into cohesive service areas.  In
>> reality, most agencies would initially deploy their API server and not
>> touch it for five years.  Two agencies offering service in the same city
>> would both provide their own API endpoints, likely with overlapping data.
>>  Imagine trying to write an app that sits on top of all that.  Imagine
>> route planning in such a world.  It's of course possible, but it doesn't
>> sound like much fun ; )
>>
>> And we don't even live in anything like that world.  In reality, agencies
>> have been deploying APIs of their own creation for years now (witness the
>> list below).  Trying to convince agencies to switch to a standards-based
>> API when they already have an API that is working and has local apps is a
>> tricky value proposition.  The value proposition is slightly better for new
>> agencies who are thinking about deploying a new API, but the "Deploy this
>> standard API and you will magically have 20 apps tomorrow" narrative
>> doesn't seem to work that way in reality quite yet.  And local developers
>> tend to work with whatever they can get, either way, so the new agencies
>> are generally fine no matter what they deploy.
>>
>> In the end, I decided that it's easiest writing apps against a single API
>> that aggregates raw transit data from lots of agencies.  As such, I went to
>> work for one of the biggest aggregators of transit data in the world
>> (ahem... Google).
>>
>> These days, I spend a lot of time advocating for standardization of bulk
>> data formats (mostly GTFS and GTFS-realtime) because it's what Google
>> needs.  But more importantly, I think the open-data movement and it's
>> developers are ultimately better served by bulk access to data than
>> high-level APIs.  When someone provides a high-level API, they are
>> necessarily making decision about what you will and won't be able to do
>> with the API.  However, with bulk data formats, you aren't so constrained.
>>  Applications like OpenTripPlanner wouldn't be possible without bulk access
>> to schedule data, for example.
>>
>> So, in summary, I'd encourage the working group to advocate for bulk data
>> formats, not high-level service APIs.
>>
>> Now all we have to do is agree on preferred bulk data formats ;)
>>
>> Finally, as promised, just to point out the size of the problem, a
>> handful of Transit Service-Oriented APIs off the top of my head in addition
>> to the ones that have already been mentioned (a bit North America centric,
>> but I see the same fragmentation happening in Europe as well):
>>
>> SIRI, epsecially with the discovery features that Michael Frumin has been
>> championing
>> NYC MTA BusTime APIs: http://bustime.mta.info/wiki/Developers/ -
>> Combination of RESTful SIRI + OneBusAway API
>> The OneBusAway REST API:
>> http://developer.onebusaway.org/modules/onebusaway-application-modules/current/api/where/index.html (caveat:
>> I wrote this one)
>> The Google Directions API:
>> https://developers.google.com/maps/documentation/directions/ (caveat: I
>> work for these guys ;)  But transit data for 850+ cities worldwide is
>> nothing to sneeze at)
>> NextBus API: http://www.nextbus.com/xmlFeedDocs/NextBusXMLFeed.pdf
>> The TFL Bus API:
>> http://www.tfl.gov.uk/assets/downloads/businessandpartners/tfl-live-bus-and-river-bus-arrivals-api-documentation(1).pdf
>> TriMet's Web Service: http://developer.trimet.org/ws_docs/
>> CTA BusTracker API:
>> http://www.transitchicago.com/developers/bustracker.aspx
>> The BART API: http://api.bart.gov/docs/overview/index.aspx
>> The MBTA API:
>> http://realtime.mbta.com/Portal/Content/Documents/MBTA-realtime_DeveloperDocumentation_v1.0.2_2013-06-25.pdf
>> The WMATA API: http://developer.wmata.com/io-docs
>> The Open 511 Data Exchange:
>> https://groups.google.com/d/msg/511sfbaydeveloperresources/zdD8ox5l2G4/yMD-Xb9R4hcJ(proposing to use SIRI + NeTex for web-services)
>>
>>
>>
>>
>>
>>
>>
>> On Sun, Jul 14, 2013 at 3:58 PM, Pieter Colpaert <
>> pieter.colpaert at okfn.org> wrote:
>>
>>> Hi all,
>>>
>>> Seems like this has been an issue for quite a while: how do we turn raw
>>> transport data into an API and which format/vocabulary should this API
>>> adhere to?
>>>
>>> * I have found a GTFS → API in nodejs over here:
>>> https://github.com/brendannee/node-gtfs
>>> * There is also http://navitia.io who has plans to go open source.
>>> * Open Trip Planner has an API as well.
>>>
>>> Are open source open transport data tools stuff that our Open Transport
>>> working group should push for? Which one would you prefer rights now?
>>>
>>> Kind regards,
>>>
>>> Pieter
>>>
>>> _______________________________________________
>>> open-transport mailing list
>>> open-transport at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/open-transport
>>> Unsubscribe: http://lists.okfn.org/mailman/options/open-transport
>>>
>>>
>>
>> _______________________________________________
>> open-transport mailing list
>> open-transport at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/open-transport
>> Unsubscribe: http://lists.okfn.org/mailman/options/open-transport
>>
>>
>
>
> --
> Stéphane Guidoin
> Director, Transportation
> Open North
> 514-862-0084
> http://opennorth.ca
> Twitter: @opennorth / @hoedic
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/open-transport/attachments/20130716/7ed3d845/attachment.html>