PyTransportNSWv2 0.8.5__tar.gz → 0.8.7__tar.gz

{PyTransportNSWv2-0.8.5 → PyTransportNSWv2-0.8.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: PyTransportNSWv2
-Version: 0.8.5
+Version: 0.8.7
 Summary: Get detailed per-trip transport information from TransportNSW
 Home-page: https://github.com/andystewart999/TransportNSW
 Author: andystewart999
@@ -148,7 +148,7 @@ Description: # TransportNSWv2
         Also note that the 'transport_type' filter, if present, only makes sure that at least **one** leg of the journey includes that transport type unless ```strict_transport_type``` is True, in which case the **first** leg must be of the requested type to be returned.
         
         ### Rate limits ###
-        By default the TransportNSW API allows each API key to make 60,000 calls in a day and up to 5 calls per second.  Usually this wouldn't be an issue but if you're requesting real-time location data for buses, because I haven't found an elegant way to map the bus route to the correct API URL (and there are about 14) it's brute-forcing it by iterating through ALL the possible URLs until it finds the appropriate journey or it runs out of URLs.  From version 0.8.2 I've added in a 0.5 second delay between loops until I can find a better approach.
+        By default the TransportNSW API allows each API key to make 60,000 calls in a day and up to 5 calls per second.  From version 0.8.7 I found a TransportNSW-maintained CSV that contains mappings of bus agency IDs to URLs so I'm using that, plus adding in a 0.75 second delay between API calls.
         
         ## Thank you
         Thank you Dav0815 for your TransportNSW library that the vast majority of this fork is based on.  I couldn't have done it without you!

{PyTransportNSWv2-0.8.5 → PyTransportNSWv2-0.8.7}/PyTransportNSWv2.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: PyTransportNSWv2
-Version: 0.8.5
+Version: 0.8.7
 Summary: Get detailed per-trip transport information from TransportNSW
 Home-page: https://github.com/andystewart999/TransportNSW
 Author: andystewart999
@@ -148,7 +148,7 @@ Description: # TransportNSWv2
         Also note that the 'transport_type' filter, if present, only makes sure that at least **one** leg of the journey includes that transport type unless ```strict_transport_type``` is True, in which case the **first** leg must be of the requested type to be returned.
         
         ### Rate limits ###
-        By default the TransportNSW API allows each API key to make 60,000 calls in a day and up to 5 calls per second.  Usually this wouldn't be an issue but if you're requesting real-time location data for buses, because I haven't found an elegant way to map the bus route to the correct API URL (and there are about 14) it's brute-forcing it by iterating through ALL the possible URLs until it finds the appropriate journey or it runs out of URLs.  From version 0.8.2 I've added in a 0.5 second delay between loops until I can find a better approach.
+        By default the TransportNSW API allows each API key to make 60,000 calls in a day and up to 5 calls per second.  From version 0.8.7 I found a TransportNSW-maintained CSV that contains mappings of bus agency IDs to URLs so I'm using that, plus adding in a 0.75 second delay between API calls.
         
         ## Thank you
         Thank you Dav0815 for your TransportNSW library that the vast majority of this fork is based on.  I couldn't have done it without you!

{PyTransportNSWv2-0.8.5 → PyTransportNSWv2-0.8.7}/README.md

@@ -140,7 +140,7 @@ Note also that the origin and destination details are just that - information ab
 Also note that the 'transport_type' filter, if present, only makes sure that at least **one** leg of the journey includes that transport type unless ```strict_transport_type``` is True, in which case the **first** leg must be of the requested type to be returned.
 
 ### Rate limits ###
-By default the TransportNSW API allows each API key to make 60,000 calls in a day and up to 5 calls per second.  Usually this wouldn't be an issue but if you're requesting real-time location data for buses, because I haven't found an elegant way to map the bus route to the correct API URL (and there are about 14) it's brute-forcing it by iterating through ALL the possible URLs until it finds the appropriate journey or it runs out of URLs.  From version 0.8.2 I've added in a 0.5 second delay between loops until I can find a better approach.
+By default the TransportNSW API allows each API key to make 60,000 calls in a day and up to 5 calls per second.  From version 0.8.7 I found a TransportNSW-maintained CSV that contains mappings of bus agency IDs to URLs so I'm using that, plus adding in a 0.75 second delay between API calls.
 
 ## Thank you
 Thank you Dav0815 for your TransportNSW library that the vast majority of this fork is based on.  I couldn't have done it without you!

{PyTransportNSWv2-0.8.5 → PyTransportNSWv2-0.8.7}/TransportNSWv2/TransportNSWv2.py

@@ -115,8 +115,8 @@ class TransportNSWv2(object):
 
             # Send the query and return an error if something goes wrong
             try:
-                response = requests.get(url, headers=header, timeout=20)
-            except:
+                response = requests.get(url, headers=header, timeout=5)
+            except Exception as ex:
                 logger.error("Network or Timeout error when calling /v1/tp/stop_finder API")
                 return None
 
@@ -157,7 +157,7 @@ class TransportNSWv2(object):
         # Send the query and return an error if something goes wrong
         # Otherwise store the response
         try:
-            response = requests.get(url, headers=header, timeout=20)
+            response = requests.get(url, headers=header, timeout=10)
         except:
             logger.error("Network or timeout error")
             return None
@@ -240,6 +240,9 @@ class TransportNSWv2(object):
                     if 'RealtimeTripId' in transportation['properties']:
                         realtimetripid = transportation['properties']['RealtimeTripId']
 
+                        # We're also going to need the agency_id
+                        agencyid = transportation['operator']['id']
+
                 # Line info
                 origin_line_name_short = "unknown"
                 if 'disassembledName' in transportation:
@@ -267,44 +270,42 @@ class TransportNSWv2(object):
                 if ((realtimetripid != 'n/a') and (self.include_realtime_location) == True):
                     # See if we can get the latitute and longitude via the Realtime Vehicle Positions API
                     # Build the URL(s) - some modes have multiple GTFS sources, unforunately
-                    url_path = self.get_url(origin_mode)
-                    url_list = self.get_url_path(origin_mode)
-                    bFoundTripID = False
-
-                    auth = 'apikey ' + self.api_key
-                    header = {'Authorization': auth}
-
-                    for mode_url in url_list:
-                        url = url_path + mode_url
-
-                        response = requests.get(url, headers=header, timeout=10)
-
-                        # Only try and process the results if we got a good return code
-                        if response.status_code == 200:
-                            # Search the feed and see if we can match realtimetripid to trip_id
-                            # If we do, capture the latitude and longitude
-                            feed = gtfs_realtime_pb2.FeedMessage()
-                            feed.ParseFromString(response.content)
-
-                            reg = re.compile(realtimetripid)
-
-                            for entity in feed.entity:
-                                if bool(re.match(reg, entity.vehicle.trip.trip_id)):
-                                    latitude = entity.vehicle.position.latitude
-                                    longitude = entity.vehicle.position.longitude
-
-                                    # We found it, so flag it and break out
-                                    bFoundTripID = True
-                                    break
 
-                        if bFoundTripID == True:
-                            # No need to look any further
-                            break
-
-                        # Put in a quick pause here to try and make sure we stay under the 5 API calls/second limit
-                        # Not usually an issue but if multiple processes are running multiple calls we might hit it
-                        time.sleep(0.5)
+                    url_base_path = self.get_base_url(origin_mode)
+                    url_mode_list = self.get_mode_list(origin_mode, agencyid)
+                    bFoundTripID = False
 
+                    #auth = 'apikey ' + self.api_key
+                    #header = {'Authorization': auth}
+
+                    if not url_mode_list is None:
+                        for mode_url in url_mode_list:
+                            url = url_base_path + mode_url
+                            response = requests.get(url, headers=header, timeout=10)
+                            # Only try and process the results if we got a good return code
+                            if response.status_code == 200:
+                                # Search the feed and see if we can match realtimetripid to trip_id
+                                # If we do, capture the latitude and longitude
+                                feed = gtfs_realtime_pb2.FeedMessage()
+                                feed.ParseFromString(response.content)
+                                reg = re.compile(realtimetripid)
+
+                                for entity in feed.entity:
+                                    if bool(re.match(reg, entity.vehicle.trip.trip_id)):
+                                        latitude = entity.vehicle.position.latitude
+                                        longitude = entity.vehicle.position.longitude
+
+                                        # We found it, so flag it and break out
+                                        bFoundTripID = True
+                                        break
+
+                            if bFoundTripID == True:
+                                # No need to look any further
+                                break
+
+                            # Put in a quick pause here to try and make sure we stay under the 5 API calls/second limit
+                            # Not usually an issue but if multiple processes are running multiple calls we might hit it
+                            time.sleep(0.75)
 
                 self.info = {
                     ATTR_DUE_IN: due,
@@ -490,7 +491,7 @@ class TransportNSWv2(object):
         }
         return modes.get(iconId, None)
 
-    def get_url(self, mode):
+    def get_base_url(self, mode):
         """Map the journey mode to the proper base real time location URL """
         v1_url = "https://api.transport.nsw.gov.au/v1/gtfs/vehiclepos"
         v2_url = "https://api.transport.nsw.gov.au/v2/gtfs/vehiclepos"
@@ -521,21 +522,50 @@ class TransportNSWv2(object):
         return alert_priorities.get(alert_priority, 99)
 
 
-    def get_url_path(self, mode):
-        """Map the journey mode to the proper modifier URL """
-        # There's probably a better way of mapping the bus route/stop/line number to the correct URL to call in the GTFS API, but as of now I don't know it... hence the brute force approach, although I hate the lack of elegance
-        bus_list = ["/buses", "/regionbuses/centralwestandorana", "/regionbuses/centralwestandorana2", "/regionbuses/newenglandnorthwest", "/regionbuses/northcoast", "/regionbuses/northcoast2", "/regionbuses/northcoast3", "/regionbuses/riverinamurray", "/regionbuses/riverinamurray2", "/regionbuses/southeasttablelands", "/regionbuses/southeasttablelands2", "/regionbuses/sydneysurrounds", "/regionbuses/newcastlehunter", "/regionbuses/farwest"]
+    def get_mode_list(self, mode, agencyid):
+        """
+        Map the journey mode to the proper modifier URL.  If the mode is Bus, Coach or School bus then use the agency ID to invoke the GTFS datastore search API
+        which will give us the appropriate URL to call later - we still have to do light rail the old-fashioned, brute-force way though
+        """
 
-        url_options = {
-            "Train"      : ["/sydneytrains"],
-            "Metro"      : ["/metro"],
-            "Light rail" : ["/lightrail/innerwest", "/lightrail/cbdandsoutheast", "/lightrail/newcastle"],
-            "Bus"        : bus_list,
-            "Coach"      : bus_list,
-            "Ferry"      : ["/ferries/sydneyferries"],
-            "School bus" : bus_list
-        }
-        return url_options.get(mode, None)
+        if mode in ["Bus", "Coach", "School bus"]:
+            # Use the API to determine the appropriate URL
+            url = "https://opendata.transport.nsw.gov.au/data/api/action/datastore_search?resource_id=30b850b7-f439-4e30-8072-e07ef62a2a36&filters={%22For%20Realtime%20GTFS%20agency_id%22:%22" + agencyid + "%22}&limit=1"
+
+            # Send the query and return an error if something goes wrong
+            try:
+                response = requests.get(url, timeout=5)
+            except Exception as ex:
+                logger.error("Error " + str(ex) + " querying GTFS URL datastore")
+                return None
+
+            # If we get bad status code, log error and return with None
+            if response.status_code != 200:
+                if response.status_code == 429:
+                    logger.error("Error " + str(response.status_code) + " calling /v1/tp/stop_finder API; rate limit exceeded")
+                else:
+                    logger.error("Error " + str(response.status_code) + " calling /v1/tp/stop_finder API; check api key")
+
+                return None
+
+            # Parse the result as JSON
+            result = response.json()
+            if 'records' in result['result'] and len(result['result']['records']) > 0:
+                mode_path = result['result']['records'][0]['For Realtime parameter']
+            else:
+                return None
+
+            bus_list = ["/" + mode_path]
+            return bus_list
+        else:
+            # Handle the other modes
+            url_options = {
+                "Train"      : ["/sydneytrains"],
+                "Metro"      : ["/metro"],
+                "Light rail" : ["/lightrail/innerwest", "/lightrail/cbdandsoutheast", "/lightrail/newcastle"],
+                "Ferry"      : ["/ferries/sydneyferries"]
+            }
+            return url_options.get(mode, None)
 
 
     def get_due(self, estimated):

{PyTransportNSWv2-0.8.5 → PyTransportNSWv2-0.8.7}/setup.py

@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
 
 setuptools.setup(
     name="PyTransportNSWv2",
-    version="0.8.5",
+    version="0.8.7",
     author="andystewart999",
     author_email="andy.stewart@live.com",
     description="Get detailed per-trip transport information from TransportNSW",