opal-server 0.7.0__tar.gz → 0.7.2__tar.gz

{opal-server-0.7.0 → opal-server-0.7.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: opal-server
-Version: 0.7.0
+Version: 0.7.2
 Summary: OPAL is an administration layer for Open Policy Agent (OPA), detecting changes to both policy and data and pushing live updates to your agents. The opal-server creates a pub/sub channel clients can subscribe to (i.e: acts as coordinator). The server also tracks a git repository (via webhook) for updates to policy (or static data) and accepts continuous data update notifications via REST api, which are then pushed to clients.
 Home-page: https://github.com/permitio/opal
 Author: Or Weis, Asaf Cohen
@@ -52,26 +52,29 @@ Open Policy Administration Layer
 <a href="https://twitter.com/intent/follow?original_referer=https%3A%2F%2Fpublish.twitter.com%2F%3FbuttonType%3DFollowButton%26query%3Dhttps%253A%252F%252Ftwitter.com%252Fpermit_io%26widget%3DButton&ref_src=twsrc%5Etfw&region=follow_link&screen_name=permit_io&tw_p=followbutton"><img src="https://img.shields.io/twitter/follow/permit_io?label=Follow%20%40permit_io&style=social">
 </a>
 
-
-
 ## What is OPAL?
 
-OPAL is an administration layer for <a href="https://www.openpolicyagent.org/">Open Policy Agent (OPA)</a>, detecting changes to both policy and policy data in realtime and pushing live updates to your agents. OPAL brings open-policy up to the speed needed by live applications.
+OPAL is an administration layer for Policy Engines such as <a target="_blank" href="https://www.openpolicyagent.org/">Open Policy Agent (OPA)</a>, and <a target="_blank" href="https://github.com/permitio/cedar-agent">AWS' Cedar Agent</a> detecting changes to both policy and policy data in realtime and pushing live updates to your agents. OPAL brings open-policy up to the speed needed by live applications.
 
 As your application state changes (whether it's via your APIs, DBs, git, S3 or 3rd-party SaaS services), OPAL will make sure your services are always in sync with the authorization data and policy they need (and only those they need).
 
-Check out our main site at <a href="https://opal.ac">OPAL.ac</a>, <a href="https://youtu.be/tG8jrdcc7Zo">this video</a> briefly explaining OPAL and how it works with OPA, and a deeper dive into it at [this OWASP DevSlop talk](https://www.youtube.com/watch?v=1_Iz0tRQCH4).
+Check out our main site at <a target="_blank" href="https://opal.ac">OPAL.ac</a>, <a target="_blank" href="https://youtu.be/tG8jrdcc7Zo">this video</a> briefly explaining OPAL and how it works with OPA, and a deeper dive into it at [this OWASP DevSlop talk](https://www.youtube.com/watch?v=1_Iz0tRQCH4).
 
 ## Why use OPAL?
 
 OPAL is the easiest way to keep your solution's authorization layer up-to-date in realtime. It aggregates policy and data from across the field and integrates them seamlessly into the authorization layer, and is microservices and cloud-native.
 
-## OPA + OPAL = 💜
+## OPA + OPAL == 💜
 
 While OPA (Open Policy Agent) decouples policy from code in a highly-performant and elegant way, the challenge of keeping policy agents up-to-date remains.
 This is especially true in applications, where each user interaction or API call may affect access-control decisions.
 OPAL runs in the background, supercharging policy-agents, keeping them in sync with events in realtime.
 
+## AWS Cedar + OPAL == 💪
+
+Cedar is a very powerful policy language, which powers AWS' AVP (Amazon Verified Permissions) - but what if you want to enjoy the power of Cedar on another cloud, locally, or on premise?
+This is where [Cedar-Agent](https://github.com/permitio/cedar-agent) and OPAL come in.
+
 ## Documentation
 
 - 📃 &nbsp; [Full documentation is available here](https://docs.opal.ac)
@@ -156,3 +159,4 @@ If you are using our project, please consider giving us a ⭐️
 ## There's more!
 
 - Check out [OPToggles](https://github.com/permitio/OPToggles), which enables you to create user targeted feature flags/toggles based on Open Policy managed authorization rules!
+- Check out [Cedar-Agent](https://github.com/permitio/cedar-agent), the easiest way to deploy & run AWS Cedar.

{opal-server-0.7.0 → opal-server-0.7.2}/opal_server/config.py

@@ -15,6 +15,11 @@ class PolicySourceTypes(str, Enum):
     Api = "API"
 
 
+class PolicyBundleServerType(str, Enum):
+    HTTP = "HTTP"
+    AWS_S3 = "AWS-S3"
+
+
 class ServerRole(str, Enum):
     Primary = "primary"
     Secondary = "secondary"
@@ -107,10 +112,21 @@ class OpalServerConfig(Confi):
     LEADER_LOCK_FILE_PATH = confi.str(
         "LEADER_LOCK_FILE_PATH", "/tmp/opal_server_leader.lock"
     )
+    POLICY_BUNDLE_SERVER_TYPE = confi.enum(
+        "POLICY_BUNDLE_SERVER_TYPE",
+        PolicyBundleServerType,
+        PolicyBundleServerType.HTTP,
+        description="The type of bundle server e.g. basic HTTP , AWS S3. (affects how we authenticate with it)",
+    )
     POLICY_BUNDLE_SERVER_TOKEN = confi.str(
         "POLICY_BUNDLE_SERVER_TOKEN",
         None,
-        description="Bearer token to sent to API bundle server",
+        description="Secret token to be sent to API bundle server",
+    )
+    POLICY_BUNDLE_SERVER_TOKEN_ID = confi.str(
+        "POLICY_BUNDLE_SERVER_TOKEN_ID",
+        None,
+        description="The id of the secret token to be sent to API bundle server",
     )
     POLICY_BUNDLE_TMP_PATH = confi.str(
         "POLICY_BUNDLE_TMP_PATH",
@@ -159,7 +175,9 @@ class OpalServerConfig(Confi):
 
     # Data updates
     ALL_DATA_TOPIC = confi.str(
-        "ALL_DATA_TOPIC", DEFAULT_DATA_TOPIC, description="Top level topic for data"
+        "ALL_DATA_TOPIC",
+        DEFAULT_DATA_TOPIC,
+        description="Top level topic for data",
     )
     ALL_DATA_ROUTE = confi.str("ALL_DATA_ROUTE", "/policy-data")
     ALL_DATA_URL = confi.str(
@@ -243,7 +261,7 @@ class OpalServerConfig(Confi):
     SERVER_PORT = confi.str(
         "SERVER_PORT",
         None,
-        # Users have expirienced errors when kubernetes sets the envar OPAL_SERVER_PORT="tcp://..." (which fails to parse as a port integer).
+        # Users have experienced errors when kubernetes sets the env-var OPAL_SERVER_PORT="tcp://..." (which fails to parse as a port integer).
         description="Deprecated, use SERVER_BIND_PORT instead",
     )
 

{opal-server-0.7.0 → opal-server-0.7.2}/opal_server/data/api.py

@@ -128,7 +128,7 @@ def init_data_updates_router(
             logger.error(f"Unauthorized to publish update: {repr(e)}")
             raise
 
-        data_update_publisher.publish_data_updates(update)
+        await data_update_publisher.publish_data_updates(update)
         return {"status": "ok"}
 
     return router

{opal-server-0.7.0 → opal-server-0.7.2}/opal_server/data/data_update_publisher.py

@@ -63,7 +63,7 @@ class DataUpdatePublisher:
 
         return topic_combos
 
-    def publish_data_updates(self, update: DataUpdate):
+    async def publish_data_updates(self, update: DataUpdate):
         """Notify OPAL subscribers of a new data update by topic.
 
         Args:
@@ -107,7 +107,9 @@ class DataUpdatePublisher:
             reason=update.reason,
             entries=logged_entries,
         )
-        self._publisher.publish(list(all_topic_combos), update)
+
+        async with self._publisher:
+            await self._publisher.publish(list(all_topic_combos), update)
 
     async def _periodic_update_callback(
         self, update: DataSourceEntryWithPollingInterval
@@ -118,7 +120,7 @@ class DataUpdatePublisher:
         )
         # Create new publish entry
 
-        return self.publish_data_updates(
+        return await self.publish_data_updates(
             DataUpdate(reason="Periodic Update", entries=[update])
         )
 

{opal-server-0.7.0 → opal-server-0.7.2}/opal_server/policy/watcher/callbacks.py

@@ -116,4 +116,7 @@ async def publish_changed_directories(
     )
 
     if notification:
-        publisher.publish(topics=notification.topics, data=notification.update.dict())
+        async with publisher:
+            await publisher.publish(
+                topics=notification.topics, data=notification.update.dict()
+            )

{opal-server-0.7.0 → opal-server-0.7.2}/opal_server/policy/watcher/factory.py

@@ -25,6 +25,8 @@ def setup_watcher_task(
     polling_interval: int = None,
     request_timeout: int = None,
     policy_bundle_token: str = None,
+    policy_bundle_token_id: str = None,
+    policy_bundle_server_type: str = None,
     extensions: Optional[List[str]] = None,
     bundle_ignore: Optional[List[str]] = None,
 ) -> BasePolicyWatcherTask:
@@ -41,6 +43,8 @@ def setup_watcher_task(
         polling_interval(int):  how many seconds need to wait between polling
         request_timeout(int):  how many seconds need to wait until timeout
         policy_bundle_token(int):  auth token to include in connections to OPAL server. Defaults to POLICY_BUNDLE_SERVER_TOKEN.
+        policy_bundle_token_id(int):  id token to include in connections to OPAL server. Defaults to POLICY_BUNDLE_SERVER_TOKEN_ID.
+        policy_bundle_server_type (str): type of policy bundle server (HTTP S3). Defaults to POLICY_BUNDLE_SERVER_TYPE
         extensions(list(str), optional):  list of extantions to check when new policy arrive default is FILTER_FILE_EXTENSIONS
         bundle_ignore(list(str), optional):  list of glob paths to use for excluding files from bundle default is OPA_BUNDLE_IGNORE
     """
@@ -105,11 +109,19 @@ def setup_watcher_task(
             logger.warning(
                 "POLICY_BUNDLE_URL is unset but policy watcher is enabled! disabling watcher."
             )
+        policy_bundle_token_id = load_conf_if_none(
+            policy_bundle_token_id, opal_server_config.POLICY_BUNDLE_SERVER_TOKEN_ID
+        )
+        policy_bundle_server_type = load_conf_if_none(
+            policy_bundle_server_type, opal_server_config.POLICY_BUNDLE_SERVER_TYPE
+        )
         watcher = ApiPolicySource(
             remote_source_url=remote_source_url,
             local_clone_path=clone_path,
             polling_interval=polling_interval,
             token=policy_bundle_token,
+            token_id=policy_bundle_token_id,
+            bundle_server_type=policy_bundle_server_type,
             policy_bundle_path=opal_server_config.POLICY_BUNDLE_TMP_PATH,
             policy_bundle_git_add_pattern=opal_server_config.POLICY_BUNDLE_GIT_ADD_PATTERN,
         )

{opal-server-0.7.0 → opal-server-0.7.2}/opal_server/policy/watcher/task.py

@@ -17,6 +17,7 @@ class BasePolicyWatcherTask:
         self._tasks: List[asyncio.Task] = []
         self._should_stop: Optional[asyncio.Event] = None
         self._pubsub_endpoint = pubsub_endpoint
+        self._webhook_tasks: List[asyncio.Task] = []
 
     async def __aenter__(self):
         await self.start()
@@ -26,8 +27,13 @@ class BasePolicyWatcherTask:
         await self.stop()
 
     async def _on_webhook(self, topic: Topic, data: Any):
-        logger.info("Webhook listener triggered")
-        self._tasks.append(asyncio.create_task(self.trigger(topic, data)))
+        logger.info(f"Webhook listener triggered ({len(self._webhook_tasks)})")
+        for task in self._webhook_tasks:
+            if task.done():
+                # Clean references to finished tasks
+                self._webhook_tasks.remove(task)
+
+        self._webhook_tasks.append(asyncio.create_task(self.trigger(topic, data)))
 
     async def _listen_to_webhook_notifications(self):
         # Webhook api route can be hit randomly in all workers, so it publishes a message to the webhook topic.
@@ -58,7 +64,7 @@ class BasePolicyWatcherTask:
     async def stop(self):
         """stops all policy watcher tasks."""
         logger.info("Stopping policy watcher")
-        for task in self._tasks:
+        for task in self._tasks + self._webhook_tasks:
             if not task.done():
                 task.cancel()
         await asyncio.gather(*self._tasks, return_exceptions=True)

{opal-server-0.7.0 → opal-server-0.7.2}/opal_server/scopes/api.py

@@ -342,10 +342,9 @@ def init_scope_router(
             for entry in update.entries:
                 entry.topics = [f"data:{topic}" for topic in entry.topics]
 
-            async with ScopedServerSideTopicPublisher(
-                pubsub_endpoint, scope_id
-            ) as publisher:
-                DataUpdatePublisher(publisher).publish_data_updates(update)
+            await DataUpdatePublisher(
+                ScopedServerSideTopicPublisher(pubsub_endpoint, scope_id)
+            ).publish_data_updates(update)
         except Unauthorized as ex:
             logger.error(f"Unauthorized to publish update: {repr(ex)}")
             raise

{opal-server-0.7.0 → opal-server-0.7.2}/opal_server/scopes/service.py

@@ -94,8 +94,7 @@ class NewCommitsCallbacks(PolicyFetcherCallbacks):
         async with ScopedServerSideTopicPublisher(
             self._pubsub_endpoint, self._scope_id
         ) as publisher:
-            publisher.publish(notification.topics, notification.update)
-            await publisher.wait()
+            await publisher.publish(notification.topics, notification.update)
 
 
 class ScopesService:

{opal-server-0.7.0 → opal-server-0.7.2}/opal_server.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: opal-server
-Version: 0.7.0
+Version: 0.7.2
 Summary: OPAL is an administration layer for Open Policy Agent (OPA), detecting changes to both policy and data and pushing live updates to your agents. The opal-server creates a pub/sub channel clients can subscribe to (i.e: acts as coordinator). The server also tracks a git repository (via webhook) for updates to policy (or static data) and accepts continuous data update notifications via REST api, which are then pushed to clients.
 Home-page: https://github.com/permitio/opal
 Author: Or Weis, Asaf Cohen
@@ -52,26 +52,29 @@ Open Policy Administration Layer
 <a href="https://twitter.com/intent/follow?original_referer=https%3A%2F%2Fpublish.twitter.com%2F%3FbuttonType%3DFollowButton%26query%3Dhttps%253A%252F%252Ftwitter.com%252Fpermit_io%26widget%3DButton&ref_src=twsrc%5Etfw&region=follow_link&screen_name=permit_io&tw_p=followbutton"><img src="https://img.shields.io/twitter/follow/permit_io?label=Follow%20%40permit_io&style=social">
 </a>
 
-
-
 ## What is OPAL?
 
-OPAL is an administration layer for <a href="https://www.openpolicyagent.org/">Open Policy Agent (OPA)</a>, detecting changes to both policy and policy data in realtime and pushing live updates to your agents. OPAL brings open-policy up to the speed needed by live applications.
+OPAL is an administration layer for Policy Engines such as <a target="_blank" href="https://www.openpolicyagent.org/">Open Policy Agent (OPA)</a>, and <a target="_blank" href="https://github.com/permitio/cedar-agent">AWS' Cedar Agent</a> detecting changes to both policy and policy data in realtime and pushing live updates to your agents. OPAL brings open-policy up to the speed needed by live applications.
 
 As your application state changes (whether it's via your APIs, DBs, git, S3 or 3rd-party SaaS services), OPAL will make sure your services are always in sync with the authorization data and policy they need (and only those they need).
 
-Check out our main site at <a href="https://opal.ac">OPAL.ac</a>, <a href="https://youtu.be/tG8jrdcc7Zo">this video</a> briefly explaining OPAL and how it works with OPA, and a deeper dive into it at [this OWASP DevSlop talk](https://www.youtube.com/watch?v=1_Iz0tRQCH4).
+Check out our main site at <a target="_blank" href="https://opal.ac">OPAL.ac</a>, <a target="_blank" href="https://youtu.be/tG8jrdcc7Zo">this video</a> briefly explaining OPAL and how it works with OPA, and a deeper dive into it at [this OWASP DevSlop talk](https://www.youtube.com/watch?v=1_Iz0tRQCH4).
 
 ## Why use OPAL?
 
 OPAL is the easiest way to keep your solution's authorization layer up-to-date in realtime. It aggregates policy and data from across the field and integrates them seamlessly into the authorization layer, and is microservices and cloud-native.
 
-## OPA + OPAL = 💜
+## OPA + OPAL == 💜
 
 While OPA (Open Policy Agent) decouples policy from code in a highly-performant and elegant way, the challenge of keeping policy agents up-to-date remains.
 This is especially true in applications, where each user interaction or API call may affect access-control decisions.
 OPAL runs in the background, supercharging policy-agents, keeping them in sync with events in realtime.
 
+## AWS Cedar + OPAL == 💪
+
+Cedar is a very powerful policy language, which powers AWS' AVP (Amazon Verified Permissions) - but what if you want to enjoy the power of Cedar on another cloud, locally, or on premise?
+This is where [Cedar-Agent](https://github.com/permitio/cedar-agent) and OPAL come in.
+
 ## Documentation
 
 - 📃 &nbsp; [Full documentation is available here](https://docs.opal.ac)
@@ -156,3 +159,4 @@ If you are using our project, please consider giving us a ⭐️
 ## There's more!
 
 - Check out [OPToggles](https://github.com/permitio/OPToggles), which enables you to create user targeted feature flags/toggles based on Open Policy managed authorization rules!
+- Check out [Cedar-Agent](https://github.com/permitio/cedar-agent), the easiest way to deploy & run AWS Cedar.

{opal-server-0.7.0 → opal-server-0.7.2}/opal_server.egg-info/requires.txt

@@ -9,12 +9,12 @@ aioredis<3,>=2.0.1
 pygit2<2,>=1.9.2
 asgiref<4,>=3.5.2
 redis<5,>=4.3.4
-opal-common==0.7.0
+opal-common==0.7.2
 charset-normalizer<3,>=2.0.12
 idna<4,>=3.3
 typer<1,>=0.4.1
 fastapi<1,>=0.78.0
-fastapi_websocket_pubsub==0.3.3
+fastapi_websocket_pubsub==0.3.4
 fastapi_websocket_rpc<1,>=0.1.21
 gunicorn<21,>=20.1.0
 pydantic[email]<2,>=1.9.1