kattis2canvas 0.1.0__tar.gz

kattis2canvas-0.1.0/PKG-INFO

@@ -0,0 +1,106 @@
+Metadata-Version: 2.4
+Name: kattis2canvas
+Version: 0.1.0
+Summary: CLI tool to integrate Kattis offerings with Canvas LMS courses
+Author: bcr33d
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/bcr33d/kattis2canvas
+Project-URL: Repository, https://github.com/bcr33d/kattis2canvas
+Keywords: kattis,canvas,lms,education,grading
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Education
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Education
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Requires-Dist: click>=8.0
+Requires-Dist: requests>=2.25
+Requires-Dist: beautifulsoup4>=4.9
+Requires-Dist: canvasapi>=2.0
+
+# kattis2canvas
+
+this is a simple python tool that uses the canvasapi toolkit to integrate a kattis offering with a canvas course. the tool was specifically made to work with the commercial kattis (as you will see in a moment). it would take a bit of tweaking on the web scraping to make it work with open.kattis.com.
+
+the kattis connection is done using web scraping and thus it is very fragile! at the end, i will highlight where it is most vulnerable.
+
+# setting up kattis2canvas
+
+## the config file
+
+first you will need to set up the config file. it has all your authorization tokens so DON'T CHECK IT IN. i specificially look for it in the app_dir as defined by click to get it far away from source. on linux this ends up being a file called ~/.config/kattis2canvas.ini. this is how it should be populated:
+
+```
+[kattis]
+username: YOUR_KATTIS_USERNAME_NOT_EMAIL
+token: SOME_RANDOM_CHARACTERS
+hostname: THE_DOMAIN_NAME_OF_YOUR_KATTIS_INSTANCE
+loginurl: THE_URL_TO_LOG_IN_TO_KATTIS
+
+[canvas]
+url=URL_OF_YOUR_CANVAS_INSTANCE
+token=SOME_RANDOM_CHARACTERS
+```
+
+you can easily get the kattis section by going to https://\<kattis>/download/kattisrc where \<kattis> is your instance of kattis. you will need to move the lines around slightly. for canvas the url is the one you use to access the main page of canvas. you generate the token in the bottom of the Account -> Settings page.
+
+you can check that everything is set up by running:
+
+```
+kattis2canvas list-offerings
+kattis2canvas list-assignments
+```
+
+or if you built the pyz file using make_zipapp.sh
+
+```
+kattis2canvas list-offerings
+kattis2canvas list-assignments
+```
+
+## mapping student kattis accounts to canvas accounts
+
+in canvas, you can associate various URLs with your account in the Links section of Account -> Profile. students need to put the URL of their kattis account in a link with the word "kattis" (in any case) in the title. this ends up being the join key for kattis2canvas.
+
+you can check if the students have set up the links properly using
+
+```
+kattis2canvas kattislinks
+```
+
+# using kattis2canvas
+
+## populating kattis assignments in canvas
+
+when you create new kattis assignments, you will need to get it mirrored into canvas. currently the **course2canvas** command will put all of the assignments into an assignment group called kattis. the assignment group must be created in your course before **course2canvas**.
+
+when you specify the names of offerings in kattis and courses in canvas, you can specify a substring of the name and **kattis2canvas** will be able to use it as long as it matches exactly one name. if it doesn't, it will show it what it found.
+
+when creating the assignment in canvas, anything you have put in the description in kattis will also be replicated in canvas along with a link to the kattis assignment.
+
+if you have made changes to a kattis assignment that you have already populated in canvas, use the --force option to force an update. (right now there isn't a way to force individual assignments.)
+
+if you use modules, you can use the **--add-to-module** flag to add the kattis assignments to a module. at this point, it puts all of the kattis assignments into that one module.
+
+## getting the results to canvas
+
+the **submissions2canvas** will replicate results from student submissions to kattis into canvas. it will only replicate results that are either better or the same as or newer than previous results it has replicated. a summary of the problem, score, and link to the submission will be added as a comment for the student in the gradebook for the relevant assignment. the idea is that when it's time to grade, you have easy access to the results and the link to the source from canvas speedgrader.
+
+# kattis webscraping
+
+unfortuately, scraping the kattis API is very adhoc and it would be naive to think that it wouldn't change in ways that will break this tool. we use BeautifulSoupe (it's pretty beautiful...) so here are the features that the script relies on for kattis webpages: (note the term "assume" is used for things we have to believe because that is the only reasonable way that we can use the information we are given.)
+
+* the list of offerings: (HOSTNAME is from the config file above) we assume http://HOSTNAME/ will give us a page will all the offerings and the urls in the href of those offerings will have the form **/courses/[^/]+/[^/]+**
+* the list of assignments: we assume the offering page will have the detail page for assignments in hrefs of anchor tabs of the form "assignments/\w+$".
+* the assignment details: we assume the assignment detail page will have an \<h2> tag with the text "Description" followed by a sibling \<p> tag that entirely contains the description. we also assume that there will be a \<td> for "start time" and another for "end time" we do case insensitve comparisons to find them. we assume that the following \<td> tage will have the time.
+* time: TIME IS HARD! if the time is recent, kattis will drop the date, so if we get a time with no date, we take the current time and up date the HH:MM:SS with the date we get from kattis. we also assume all dates are UTC.
+* getting submissions: we assume the submissions for an assignment are found at https://HOSTNAME/OFFERING/assignments/ASSIGNMENT_ID/submissions it appears that all submissions for the assignment are listed on that page. submissions for a problem outside of the assignment time period will not show up on that page. the submissions are in a table called "judege"table". the headers \<th> that we are looking for are "User" for the user url, "Problem" for the name of the problem, "Test cases" for the score reflected as sucess/count with -/- indicating no tries, and "" indicating the header for the url of the submission. once we know the column numbers we want, we have to look for the \<tbody> child of the table (problems happen if you try to look for \<td> recursively from the table!) then we look for \<tr> children of \<tbody> which have a "data-submission-id" attribute.
+
+

kattis2canvas-0.1.0/README.md

@@ -0,0 +1,79 @@
+# kattis2canvas
+
+this is a simple python tool that uses the canvasapi toolkit to integrate a kattis offering with a canvas course. the tool was specifically made to work with the commercial kattis (as you will see in a moment). it would take a bit of tweaking on the web scraping to make it work with open.kattis.com.
+
+the kattis connection is done using web scraping and thus it is very fragile! at the end, i will highlight where it is most vulnerable.
+
+# setting up kattis2canvas
+
+## the config file
+
+first you will need to set up the config file. it has all your authorization tokens so DON'T CHECK IT IN. i specificially look for it in the app_dir as defined by click to get it far away from source. on linux this ends up being a file called ~/.config/kattis2canvas.ini. this is how it should be populated:
+
+```
+[kattis]
+username: YOUR_KATTIS_USERNAME_NOT_EMAIL
+token: SOME_RANDOM_CHARACTERS
+hostname: THE_DOMAIN_NAME_OF_YOUR_KATTIS_INSTANCE
+loginurl: THE_URL_TO_LOG_IN_TO_KATTIS
+
+[canvas]
+url=URL_OF_YOUR_CANVAS_INSTANCE
+token=SOME_RANDOM_CHARACTERS
+```
+
+you can easily get the kattis section by going to https://\<kattis>/download/kattisrc where \<kattis> is your instance of kattis. you will need to move the lines around slightly. for canvas the url is the one you use to access the main page of canvas. you generate the token in the bottom of the Account -> Settings page.
+
+you can check that everything is set up by running:
+
+```
+kattis2canvas list-offerings
+kattis2canvas list-assignments
+```
+
+or if you built the pyz file using make_zipapp.sh
+
+```
+kattis2canvas list-offerings
+kattis2canvas list-assignments
+```
+
+## mapping student kattis accounts to canvas accounts
+
+in canvas, you can associate various URLs with your account in the Links section of Account -> Profile. students need to put the URL of their kattis account in a link with the word "kattis" (in any case) in the title. this ends up being the join key for kattis2canvas.
+
+you can check if the students have set up the links properly using
+
+```
+kattis2canvas kattislinks
+```
+
+# using kattis2canvas
+
+## populating kattis assignments in canvas
+
+when you create new kattis assignments, you will need to get it mirrored into canvas. currently the **course2canvas** command will put all of the assignments into an assignment group called kattis. the assignment group must be created in your course before **course2canvas**.
+
+when you specify the names of offerings in kattis and courses in canvas, you can specify a substring of the name and **kattis2canvas** will be able to use it as long as it matches exactly one name. if it doesn't, it will show it what it found.
+
+when creating the assignment in canvas, anything you have put in the description in kattis will also be replicated in canvas along with a link to the kattis assignment.
+
+if you have made changes to a kattis assignment that you have already populated in canvas, use the --force option to force an update. (right now there isn't a way to force individual assignments.)
+
+if you use modules, you can use the **--add-to-module** flag to add the kattis assignments to a module. at this point, it puts all of the kattis assignments into that one module.
+
+## getting the results to canvas
+
+the **submissions2canvas** will replicate results from student submissions to kattis into canvas. it will only replicate results that are either better or the same as or newer than previous results it has replicated. a summary of the problem, score, and link to the submission will be added as a comment for the student in the gradebook for the relevant assignment. the idea is that when it's time to grade, you have easy access to the results and the link to the source from canvas speedgrader.
+
+# kattis webscraping
+
+unfortuately, scraping the kattis API is very adhoc and it would be naive to think that it wouldn't change in ways that will break this tool. we use BeautifulSoupe (it's pretty beautiful...) so here are the features that the script relies on for kattis webpages: (note the term "assume" is used for things we have to believe because that is the only reasonable way that we can use the information we are given.)
+
+* the list of offerings: (HOSTNAME is from the config file above) we assume http://HOSTNAME/ will give us a page will all the offerings and the urls in the href of those offerings will have the form **/courses/[^/]+/[^/]+**
+* the list of assignments: we assume the offering page will have the detail page for assignments in hrefs of anchor tabs of the form "assignments/\w+$".
+* the assignment details: we assume the assignment detail page will have an \<h2> tag with the text "Description" followed by a sibling \<p> tag that entirely contains the description. we also assume that there will be a \<td> for "start time" and another for "end time" we do case insensitve comparisons to find them. we assume that the following \<td> tage will have the time.
+* time: TIME IS HARD! if the time is recent, kattis will drop the date, so if we get a time with no date, we take the current time and up date the HH:MM:SS with the date we get from kattis. we also assume all dates are UTC.
+* getting submissions: we assume the submissions for an assignment are found at https://HOSTNAME/OFFERING/assignments/ASSIGNMENT_ID/submissions it appears that all submissions for the assignment are listed on that page. submissions for a problem outside of the assignment time period will not show up on that page. the submissions are in a table called "judege"table". the headers \<th> that we are looking for are "User" for the user url, "Problem" for the name of the problem, "Test cases" for the score reflected as sucess/count with -/- indicating no tries, and "" indicating the header for the url of the submission. once we know the column numbers we want, we have to look for the \<tbody> child of the table (problems happen if you try to look for \<td> recursively from the table!) then we look for \<tr> children of \<tbody> which have a "data-submission-id" attribute.
+
+

kattis2canvas-0.1.0/pyproject.toml

@@ -0,0 +1,44 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "kattis2canvas"
+version = "0.1.0"
+description = "CLI tool to integrate Kattis offerings with Canvas LMS courses"
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.8"
+authors = [
+    {name = "bcr33d"}
+]
+keywords = ["kattis", "canvas", "lms", "education", "grading"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Environment :: Console",
+    "Intended Audience :: Education",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Education",
+]
+dependencies = [
+    "click>=8.0",
+    "requests>=2.25",
+    "beautifulsoup4>=4.9",
+    "canvasapi>=2.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/bcr33d/kattis2canvas"
+Repository = "https://github.com/bcr33d/kattis2canvas"
+
+[project.scripts]
+kattis2canvas = "kattis2canvas.cli:top"
+
+[tool.setuptools.packages.find]
+where = ["src"]

kattis2canvas-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

kattis2canvas-0.1.0/src/kattis2canvas/__init__.py

@@ -0,0 +1,3 @@
+"""kattis2canvas - CLI tool to integrate Kattis offerings with Canvas LMS courses."""
+
+__version__ = "0.1.0"

kattis2canvas-0.1.0/src/kattis2canvas/__main__.py

@@ -0,0 +1,6 @@
+"""Entry point for python -m kattis2canvas."""
+
+from kattis2canvas.cli import top
+
+if __name__ == "__main__":
+    top()

kattis2canvas-0.1.0/src/kattis2canvas/cli.py

@@ -0,0 +1,591 @@
+import collections
+import concurrent.futures
+import configparser
+import datetime
+import os
+import re
+import sys
+from fractions import Fraction
+from typing import NamedTuple, Optional
+
+import click
+import pprint
+import requests
+import requests.cookies
+import requests.exceptions
+from bs4 import BeautifulSoup
+from canvasapi import Canvas, module
+from canvasapi.course import Course
+from canvasapi.user import User
+
+HEADERS = {'User-Agent': 'kattis-to-canvas'}
+
+
+class Config(NamedTuple):
+    kattis_username: str
+    kattis_token: str
+    kattis_loginurl: str
+    kattis_hostname: str
+    canvas_url: str
+    canvas_token: str
+
+
+config: Optional[Config] = None
+login_cookies: Optional[requests.cookies.RequestsCookieJar] = None
+
+
+class Student(NamedTuple):
+    kattis_url: str
+    name: str
+    email: str
+    canvas_id: str
+
+
+class Submission(NamedTuple):
+    user: str
+    problem: str
+    score: float
+    url: str
+    date: datetime.datetime
+
+
+now = datetime.datetime.now(datetime.timezone.utc)
+
+
+def error(message: str):
+    click.echo(click.style(message, fg='red'))
+
+
+def info(message: str):
+    click.echo(click.style(message, fg='blue'))
+
+
+def warn(message: str):
+    click.echo(click.style(message, fg='yellow'))
+
+
+def check_status(rsp: requests.Response):
+    if rsp.status_code != 200:
+        error(f"got status {rsp.status_code} for {rsp.url}.")
+        exit(6)
+
+
+# return the last element of a URL
+def extract_last(pathish: str) -> str:
+    last_slash = pathish.rindex("/")
+    if last_slash:
+        pathish = pathish[last_slash + 1:]
+    return pathish
+
+
+# for debugging
+def introspect(o):
+    print("class", o.__class__)
+    for i in dir(o):
+        print(i)
+
+
+def web_get(url: str) -> requests.Response:
+    rsp: requests.Response = requests.get(url, cookies=login_cookies, headers=HEADERS)
+    check_status(rsp)
+    return rsp
+
+
+@click.group()
+def top():
+    config_ini = click.get_app_dir("kattis2canvas.ini")
+    parser = configparser.ConfigParser()
+    parser.read([config_ini])
+    global config
+    try:
+        config = Config(
+            kattis_username=parser['kattis']['username'],
+            kattis_token=parser['kattis']['token'],
+            kattis_hostname=parser['kattis']['hostname'],
+            kattis_loginurl=parser['kattis']['loginurl'],
+            canvas_url=parser['canvas']['url'],
+            canvas_token=parser['canvas']['token'],
+        )
+    except:
+        print(f"""problem getting configuration from {config_ini}. should have the following lines:
+
+[kattis]
+username=kattis_username
+token=kattis_token
+hostname: something_like_sjsu.kattis.com
+loginurl: https://something_like_sjsu.kattis.com
+[canvas]
+url=https://something_like_sjsu.instructure.com
+token=convas_token
+""")
+        exit(2)
+
+    global login_cookies
+    args = {'user': config.kattis_username, 'script': 'true', 'token': config.kattis_token}
+    rsp = requests.post(config.kattis_loginurl, data=args, headers=HEADERS)
+    if rsp.status_code != 200:
+        error(f"Kattis login failed. Status: {rsp.status_code}")
+        exit(2)
+    login_cookies = rsp.cookies
+
+
+def get_offerings(offering_pattern: str) -> str:
+    rsp = web_get(f"https://{config.kattis_hostname}/")
+    bs = BeautifulSoup(rsp.content, 'html.parser')
+    for a in bs.find_all('a'):
+        h = a.get('href')
+        if h and re.match("/courses/[^/]+/[^/]+", h) and offering_pattern in h:
+            yield h
+
+
+@top.command()
+@click.argument("name", default="")
+def list_offerings(name: str):
+    """
+    list the possible offerings.
+    :param name: a substring of the offering name
+    """
+
+    for offering in get_offerings(name):
+        info(str(offering))
+
+
+# reformat kattis date format to canvas format
+def extract_kattis_date(element: str) -> str:
+    if element == "infinity":
+        element = "2100-01-01 00:00 UTC"
+    return datetime.datetime.strftime(datetime.datetime.strptime(element, "%Y-%m-%d %H:%M %Z"), "%Y-%m-%dT%H:%M:00%z")
+
+
+# convert canvas UTC to datetime
+def extract_canvas_date(element: str) -> datetime.datetime:
+    return datetime.datetime.strptime(element, "%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=datetime.timezone.utc)
+
+
+class Assignment(NamedTuple):
+    url: str
+    assignment_id: str
+    title: str
+    description: str
+    start: str
+    end: str
+
+
+def get_assignments(offering: str) -> [Assignment]:
+    rsp = web_get(f"https://{config.kattis_hostname}{offering}")
+    bs = BeautifulSoup(rsp.content, 'html.parser')
+    for a in bs.find_all('a'):
+        h = a.get('href')
+        if h and re.search(r"assignments/\w+$", h):
+            url = f"https://{config.kattis_hostname}{h}"
+            rsp2 = web_get(url)
+            bs2 = BeautifulSoup(rsp2.content, 'html.parser')
+            description_h2 = bs2.find("h2", string="Description", recursive=True)
+            description = None
+            if description_h2:
+                p = description_h2.find_next_sibling("p")
+                if p:
+                    description = p.text
+            all_td = iter(bs2.find_all("td"))
+            start = None
+            end = None
+            for td in all_td:
+                if td.get_text(strip=True).casefold() == "start time".casefold():
+                    start = extract_kattis_date(next(all_td).get_text(strip=True))
+                if td.get_text(strip=True).casefold() == "end time".casefold():
+                    end = extract_kattis_date(next(all_td).get_text(strip=True))
+            yield (Assignment(
+                url=url, assignment_id=url[url.rindex('/') + 1:], title=a.getText(),
+                description=description, start=start, end=end
+            ))
+
+
+@top.command()
+@click.argument("offering", default="")
+def list_assignments(offering):
+    """
+    list the assignments for the given offering.
+    :param offering: a substring of the offering name
+    """
+    for offering in get_offerings(offering):
+        for assignment in get_assignments(offering):
+            info(
+                f"{assignment.title}: {assignment.start} to {assignment.end} {assignment.description} {assignment.url}")
+
+
+@top.command()
+@click.argument("offering")
+@click.argument("assignment")
+def download_submissions(offering, assignment):
+    """
+    download the submissions for an assignment in an offering. offerings and assignments that have the given substring
+    will match.
+    """
+    for o in get_offerings(offering):
+        for a in get_assignments(o):
+            if assignment in a.title:
+                for student, probs in get_best_submissions(o, a.assignment_id).items():
+                    for problem, submission in probs.items():
+                       base_path = f"{offering}/{assignment}/{problem}/{student}"
+                       os.makedirs(base_path, exist_ok=True)
+                       rsp, name = download_submission(submission.url)
+                       with open(base_path + "/" + name, "wb") as f:
+                           f.write(rsp.content)
+
+
+def download_submission(url):
+    rsp = web_get(f"https://{config.kattis_hostname}{url}")
+    bs = BeautifulSoup(rsp.content, 'html.parser')
+    src_div = bs.find(class_="file_source-content-file", recursive=True)
+    a = src_div.find("a", recursive=True)
+    h3 = src_div.find("h3")
+    name = os.path.basename(h3.get_text().strip())
+    sanitize(name)
+    return web_get(f"https://{config.kattis_hostname}{a.get('href')}"), name
+
+
+def sanitize(name):
+    return re.sub(r"[^\w.]", "_", name)
+
+def get_course(canvas, name, is_active=True) -> Course:
+    """ find one course based on partial match """
+    course_list = get_courses(canvas, name, is_active)
+    if len(course_list) == 0:
+        error(f'no courses found that contain {name}. options are:')
+        for c in get_courses(canvas, "", is_active):
+            error(fr"    {c.name}")
+        sys.exit(2)
+    elif len(course_list) > 1:
+        error(f"multiple courses found for {name}:")
+        for c in course_list:
+            error(f"    {c.name}")
+        sys.exit(2)
+    return course_list[0]
+
+
+def get_courses(canvas: Canvas, name: str, is_active=True, is_finished=False) -> [Course]:
+    """ find the courses based on partial match """
+    courses = canvas.get_courses(enrollment_type="teacher")
+    course_list = []
+    for c in courses:
+        start = c.start_at_date if hasattr(c, "start_at_date") else now
+        end = c.end_at_date if hasattr(c, "end_at_date") else now
+        if is_active and (start > now or end < now):
+            continue
+        if is_finished and end >= now:
+            continue
+        if name in c.name:
+            c.start = start
+            c.end = end
+            course_list.append(c)
+    return course_list
+
+
+@top.command()
+@click.argument("offering")
+@click.argument("canvas_course")
+@click.option("--dryrun/--no-dryrun", default=True, help="show planned actions, do not make them happen.")
+@click.option("--force/--no-force", default=False, help="force an update of an assignment if it already exists.")
+@click.option("--add-to-module", help="the module to add the assignment to.")
+def course2canvas(offering, canvas_course, dryrun, force, add_to_module):
+    """
+    create assignments in canvas for all the assignments in kattis.
+    """
+    offerings = list(get_offerings(offering))
+    if len(offerings) == 0:
+        error(f"no offerings found for {offering}")
+        exit(3)
+    elif len(offerings) > 1:
+        error(f"multiple offerings found for {offering}: {', '.join(offerings)}")
+        exit(3)
+
+    canvas = Canvas(config.canvas_url, config.canvas_token)
+    course = get_course(canvas, canvas_course)
+
+    kattis_group = None
+    for ag in course.get_assignment_groups():
+        if ag.name == 'kattis':
+            kattis_group = ag
+            break
+    if not kattis_group:
+        # create assignment group if not present on canvas
+        if dryrun:
+            info(f"would create assignment group {kattis_group}.")
+        else:
+            kattis_group = course.create_assignment_group(name='kattis')
+            info(f"created assignment group {kattis_group}.")
+
+    if add_to_module:
+        modules = {m.name: m for m in course.get_modules()}
+        if add_to_module in modules:
+            add_to_module = modules[add_to_module]
+        else:
+            if dryrun:
+                info(f"would create and publish {add_to_module}.")
+            else:
+                args = {"name": add_to_module}
+                add_to_module = course.create_module(module=args)
+                info(f"created module {add_to_module}.")
+
+                args = {'published': "true"}
+                add_to_module.edit(module=args)
+                info(f"published module {add_to_module}.")
+
+    canvas_assignments = {a.name: a for a in course.get_assignments(assignment_group_id=kattis_group.id)}
+
+    # make sure assignments are in place
+    sorted_assignments = list(get_assignments(offerings[0]))
+    sorted_assignments.sort(key=lambda a: a.start)
+    for assignment in sorted_assignments:
+        description = assignment.description if assignment.description else ""
+        if assignment.title in canvas_assignments:
+            info(f"{assignment.title} already exists.")
+            if force:
+                if dryrun:
+                    info(f"would update {assignment.title}.")
+                else:
+                    canvas_assignments[assignment.title].edit(assignment={
+                        'assignment_group_id': kattis_group.id,
+                        'name': assignment.title,
+                        'description': f'Solve the problems found at <a href="{assignment.url}">{assignment.url}</a>. {description}',
+                        'points_possible': 100,
+                        'due_at': assignment.end,
+                        'lock_at': assignment.end,
+                        'unlock_at': assignment.start,
+                        'published': True,
+                    })
+                    info(f"updated {assignment.title}.")
+        else:
+            if dryrun:
+                info(f"would create {assignment}")
+            elif 'late' in assignment.title and assignment.title.replace("-late", "") in canvas_assignments:
+                info(f"no new assignment created as --late assignment for {assignment.title.replace('-late', '')}.")
+                continue
+            else:
+                canvas_assignments[assignment.title] = course.create_assignment({
+                    'assignment_group_id': kattis_group.id,
+                    'name': assignment.title,
+                    'description': f'Solve the problems found at <a href="{assignment.url}">{assignment.url}</a>. {description}',
+                    'points_possible': 100,
+                    'due_at': assignment.end,
+                    'lock_at': assignment.end,
+                    'unlock_at': assignment.start,
+                    'published': True,
+                })
+                info(f"created {assignment.title}.")
+        if add_to_module:
+            if assignment.title not in [i.title for i in add_to_module.get_module_items()]:
+                add_to_module.create_module_item(module_item={
+                    'title': assignment.title,
+                    'type': 'Assignment',
+                    'content_id': canvas_assignments[assignment.title].id,
+                })
+                info(f'{assignment.title} added to {add_to_module.name}')
+            else:
+                info(f'{assignment.title} already in {add_to_module.name}')
+
+
+def is_student_enrollment(user: User):
+    return "StudentEnrollment" in [e['type'] for e in user.enrollments]
+
+
+def find_kattis_link(profile: dict) -> str:
+    kattis_url = None
+    for link in profile["links"]:
+        if "kattis" in link["title"].lower():
+            kattis_url = link["url"]
+    return kattis_url
+
+
+class KattisLink(NamedTuple):
+    canvas_user: User
+    kattis_user: str
+
+
+def get_kattis_links(course: Course) -> [KattisLink]:
+    # this is so terribly slow because of all the requests, we need threads
+    with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor:
+        futures = []
+        for u in course.get_users(include=["enrollments"]):
+            if "StudentEnrollment" not in [e['type'] for e in u.enrollments]:
+                continue
+
+            def get_profile(user: User) -> Optional[KattisLink]:
+                profile = user.get_profile(include=["links"])
+                kattis_url = find_kattis_link(profile)
+                kattis_url = extract_last(kattis_url) if kattis_url else None
+                return KattisLink(canvas_user=user, kattis_user=kattis_url)
+
+            futures.append(executor.submit(get_profile, u))
+
+        links = [f.result() for f in futures if not None]
+        links.sort(key=lambda l: l.canvas_user.name)
+        return links
+
+
+@top.command()
+@click.argument("canvas_course")
+def kattislinks(canvas_course):
+    """
+    list the students in the class with their email and kattis links.
+    """
+    canvas = Canvas(config.canvas_url, config.canvas_token)
+    course = get_course(canvas, canvas_course)
+
+    for link in get_kattis_links(course):
+        if not is_student_enrollment(link.canvas_user):
+            continue
+        if link.kattis_user:
+            info(f"{link.canvas_user.name}\t{link.canvas_user.email}\t{link.kattis_user}")
+        else:
+            error(f"{link.canvas_user.name}\t{link.canvas_user.email} missing kattis link")
+
+
+@top.command()
+@click.argument("offering")
+@click.argument("canvas_course")
+@click.option("--dryrun/--no-dryrun", default=True, help="show planned actions, do not make them happen.")
+def submissions2canvas(offering, canvas_course, dryrun):
+    """
+    mirror summary of submission from kattis into canvas as a submission comment.
+    """
+    offerings = list(get_offerings(offering))
+    if len(offerings) == 0:
+        error(f"no offerings found for {offering}")
+        exit(3)
+    elif len(offerings) > 1:
+        error(f"multiple offerings found for {offering}: {', '.join(offerings)}")
+        exit(3)
+
+    canvas = Canvas(config.canvas_url, config.canvas_token)
+    course = get_course(canvas, canvas_course)
+
+    kattis_user2canvas_id = {}
+    canvas_id2kattis_user = {}
+    for link in get_kattis_links(course):
+        if link.kattis_user:
+            kattis_user2canvas_id[link.kattis_user] = link.canvas_user
+            canvas_id2kattis_user[link.canvas_user.id] = link.kattis_user
+        else:
+            warn(f"kattis link missing for {link.canvas_user.name} {link.canvas_user.email}.")
+
+    kattis_group = None
+    for ag in course.get_assignment_groups():
+        if ag.name == 'kattis':
+            kattis_group = ag
+            break
+
+    if not kattis_group:
+        error(f"no kattis assignment group in {canvas_course}")
+        exit(4)
+
+    assignments = {a.name: a for a in course.get_assignments(assignment_group_id=kattis_group.id)}
+
+    for assignment in get_assignments(offerings[0]):
+        if assignment.title.replace("-late", "") not in assignments:
+            error(f"{assignment.title.replace('-late', '')} not in canvas {canvas_course}")
+        else:
+            prefix = "LATE: " if "late" in assignment.title else ""
+            best_submissions = get_best_submissions(offering=offerings[0],
+                                                    assignment_id=assignment.assignment_id)
+            canvas_assignment = assignments[assignment.title.replace("-late", "")]
+            # find the last submissions and only add a submission if the best submission is after latest
+            submissions_by_user = {}
+            for canvas_submission in canvas_assignment.get_submissions(include=["submission_comments"]):
+                if canvas_submission.user_id in canvas_id2kattis_user:
+                    if canvas_submission.user_id in submissions_by_user:
+                        warn(
+                            f'duplicate submission for {kattis_user2canvas_id[canvas_submission.user_id]} in {assignment.title}')
+                    submissions_by_user[canvas_id2kattis_user[canvas_submission.user_id]] = canvas_submission
+                    last_comment = datetime.datetime.fromordinal(1).replace(tzinfo=datetime.timezone.utc)
+                    if canvas_submission.submission_comments:
+                        for comment in canvas_submission.submission_comments:
+                            created_at = extract_canvas_date(comment['created_at'])
+                            if created_at > last_comment:
+                                last_comment = created_at
+                    canvas_submission.last_comment = last_comment
+
+            for user, best in best_submissions.items():
+                for kattis_submission in best.values():
+                    if user not in submissions_by_user:
+                        warn(f"i don't see a canvas submission for {user}")
+                    elif user not in kattis_user2canvas_id:
+                        warn(f'skipping submission for unknown user {user}')
+                    elif kattis_submission.date > submissions_by_user[user].last_comment:
+                        if dryrun:
+                            warn(
+                                f"would update {kattis_user2canvas_id[kattis_submission.user]} on problem {kattis_submission.problem} scored {kattis_submission.score}")
+                        else:
+                            submissions_by_user[user].edit(comment={
+                                'text_comment': f"{prefix}Submission https://{config.kattis_hostname}{kattis_submission.url} scored {kattis_submission.score} on {kattis_submission.problem}."})
+                            info(
+                                f"updated {submissions_by_user[user]} {kattis_user2canvas_id[kattis_submission.user]} for {assignment.title}")
+                    else:
+                        info(f"{user} up to date")
+
+
+def get_best_submissions(offering: str, assignment_id: str) -> {str: {str: Submission}}:
+    best_submissions = collections.defaultdict(dict)
+    rsp = web_get(f"https://{config.kattis_hostname}{offering}/assignments/{assignment_id}/submissions")
+    bs = BeautifulSoup(rsp.content, "html.parser")
+    judge_table = bs.find("table", id="judge_table")
+    headers = [x.get_text().strip() for x in judge_table.find_all("th")]
+    tbody = judge_table.find("tbody")
+    for submissions in tbody.find_all("tr", recursive=False):
+        if not submissions.get("data-submission-id"):
+            continue
+        submissions = submissions.find_all("td", recursive=False)
+        if not submissions:
+            continue
+        props = {}
+        for index, td in enumerate(submissions):
+            a = td.find("a")
+            props[headers[index]] = a.get("href") if a else td.get_text().strip()
+        date = props["Date"]
+        if "-" in date:
+            date = datetime.datetime.strptime(date, "%Y-%m-%d %H:%M:%S").replace(tzinfo=now.tzinfo)
+        else:
+            hms = datetime.datetime.strptime(date, "%H:%M:%S")
+            date = now.replace(hour=hms.hour, minute=hms.minute, second=hms.second)
+            # it's not clear when the short date version is used. it might be used when it is less than 24 hours,
+            # in which case, just setting the time will make the date 24 hours more than it should be
+            if date > now:
+                date -= datetime.timedelta(days=1)
+
+        score = 0.0 if props["Test cases"] == "-/-" else float(Fraction(props["Test cases"])) * 100
+        submission = Submission(user=extract_last(props["User"]), problem=extract_last(props["Problem"]), date=date,
+                                score=score, url=props[""])
+        if submission.problem not in best_submissions[submission.user]:
+            best_submissions[submission.user] = {submission.problem: submission}
+        else:
+            current_best = best_submissions[submission.user][submission.problem]
+            if current_best.score < submission.score or (
+                    current_best.score == submission.score and current_best.date < submission.date):
+                best_submissions[submission.user][submission.problem] = submission
+    return best_submissions
+
+
+@top.command()
+@click.argument("canvas_course")
+def sendemail(canvas_course):
+    """
+    Email students if they don't have a kattis link in their profile.
+    It takes one input argument canvas course name.
+    """
+    canvas = Canvas(config.canvas_url, config.canvas_token)
+    course = get_course(canvas, canvas_course)
+
+    for link in get_kattis_links(course):
+        if not is_student_enrollment(link.canvas_user):
+            continue
+        if not link.kattis_user:
+            canvas.create_conversation(recipients=link.canvas_user.id,
+                                       body="Hello " + link.canvas_user.name + "\n\n\n Please add the missing kattis "
+                                                                               "link in bio for "
+                                                                               "course " + canvas_course + ".",
+                                       subject='Reminder: Add kattis link in profile')
+            info(f"Able to send conversation to : {link.canvas_user.id}")
+
+
+if __name__ == "__main__":
+    top()

kattis2canvas-0.1.0/src/kattis2canvas.egg-info/PKG-INFO

@@ -0,0 +1,106 @@
+Metadata-Version: 2.4
+Name: kattis2canvas
+Version: 0.1.0
+Summary: CLI tool to integrate Kattis offerings with Canvas LMS courses
+Author: bcr33d
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/bcr33d/kattis2canvas
+Project-URL: Repository, https://github.com/bcr33d/kattis2canvas
+Keywords: kattis,canvas,lms,education,grading
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Education
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Education
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Requires-Dist: click>=8.0
+Requires-Dist: requests>=2.25
+Requires-Dist: beautifulsoup4>=4.9
+Requires-Dist: canvasapi>=2.0
+
+# kattis2canvas
+
+this is a simple python tool that uses the canvasapi toolkit to integrate a kattis offering with a canvas course. the tool was specifically made to work with the commercial kattis (as you will see in a moment). it would take a bit of tweaking on the web scraping to make it work with open.kattis.com.
+
+the kattis connection is done using web scraping and thus it is very fragile! at the end, i will highlight where it is most vulnerable.
+
+# setting up kattis2canvas
+
+## the config file
+
+first you will need to set up the config file. it has all your authorization tokens so DON'T CHECK IT IN. i specificially look for it in the app_dir as defined by click to get it far away from source. on linux this ends up being a file called ~/.config/kattis2canvas.ini. this is how it should be populated:
+
+```
+[kattis]
+username: YOUR_KATTIS_USERNAME_NOT_EMAIL
+token: SOME_RANDOM_CHARACTERS
+hostname: THE_DOMAIN_NAME_OF_YOUR_KATTIS_INSTANCE
+loginurl: THE_URL_TO_LOG_IN_TO_KATTIS
+
+[canvas]
+url=URL_OF_YOUR_CANVAS_INSTANCE
+token=SOME_RANDOM_CHARACTERS
+```
+
+you can easily get the kattis section by going to https://\<kattis>/download/kattisrc where \<kattis> is your instance of kattis. you will need to move the lines around slightly. for canvas the url is the one you use to access the main page of canvas. you generate the token in the bottom of the Account -> Settings page.
+
+you can check that everything is set up by running:
+
+```
+kattis2canvas list-offerings
+kattis2canvas list-assignments
+```
+
+or if you built the pyz file using make_zipapp.sh
+
+```
+kattis2canvas list-offerings
+kattis2canvas list-assignments
+```
+
+## mapping student kattis accounts to canvas accounts
+
+in canvas, you can associate various URLs with your account in the Links section of Account -> Profile. students need to put the URL of their kattis account in a link with the word "kattis" (in any case) in the title. this ends up being the join key for kattis2canvas.
+
+you can check if the students have set up the links properly using
+
+```
+kattis2canvas kattislinks
+```
+
+# using kattis2canvas
+
+## populating kattis assignments in canvas
+
+when you create new kattis assignments, you will need to get it mirrored into canvas. currently the **course2canvas** command will put all of the assignments into an assignment group called kattis. the assignment group must be created in your course before **course2canvas**.
+
+when you specify the names of offerings in kattis and courses in canvas, you can specify a substring of the name and **kattis2canvas** will be able to use it as long as it matches exactly one name. if it doesn't, it will show it what it found.
+
+when creating the assignment in canvas, anything you have put in the description in kattis will also be replicated in canvas along with a link to the kattis assignment.
+
+if you have made changes to a kattis assignment that you have already populated in canvas, use the --force option to force an update. (right now there isn't a way to force individual assignments.)
+
+if you use modules, you can use the **--add-to-module** flag to add the kattis assignments to a module. at this point, it puts all of the kattis assignments into that one module.
+
+## getting the results to canvas
+
+the **submissions2canvas** will replicate results from student submissions to kattis into canvas. it will only replicate results that are either better or the same as or newer than previous results it has replicated. a summary of the problem, score, and link to the submission will be added as a comment for the student in the gradebook for the relevant assignment. the idea is that when it's time to grade, you have easy access to the results and the link to the source from canvas speedgrader.
+
+# kattis webscraping
+
+unfortuately, scraping the kattis API is very adhoc and it would be naive to think that it wouldn't change in ways that will break this tool. we use BeautifulSoupe (it's pretty beautiful...) so here are the features that the script relies on for kattis webpages: (note the term "assume" is used for things we have to believe because that is the only reasonable way that we can use the information we are given.)
+
+* the list of offerings: (HOSTNAME is from the config file above) we assume http://HOSTNAME/ will give us a page will all the offerings and the urls in the href of those offerings will have the form **/courses/[^/]+/[^/]+**
+* the list of assignments: we assume the offering page will have the detail page for assignments in hrefs of anchor tabs of the form "assignments/\w+$".
+* the assignment details: we assume the assignment detail page will have an \<h2> tag with the text "Description" followed by a sibling \<p> tag that entirely contains the description. we also assume that there will be a \<td> for "start time" and another for "end time" we do case insensitve comparisons to find them. we assume that the following \<td> tage will have the time.
+* time: TIME IS HARD! if the time is recent, kattis will drop the date, so if we get a time with no date, we take the current time and up date the HH:MM:SS with the date we get from kattis. we also assume all dates are UTC.
+* getting submissions: we assume the submissions for an assignment are found at https://HOSTNAME/OFFERING/assignments/ASSIGNMENT_ID/submissions it appears that all submissions for the assignment are listed on that page. submissions for a problem outside of the assignment time period will not show up on that page. the submissions are in a table called "judege"table". the headers \<th> that we are looking for are "User" for the user url, "Problem" for the name of the problem, "Test cases" for the score reflected as sucess/count with -/- indicating no tries, and "" indicating the header for the url of the submission. once we know the column numbers we want, we have to look for the \<tbody> child of the table (problems happen if you try to look for \<td> recursively from the table!) then we look for \<tr> children of \<tbody> which have a "data-submission-id" attribute.
+
+

kattis2canvas-0.1.0/src/kattis2canvas.egg-info/SOURCES.txt

@@ -0,0 +1,11 @@
+README.md
+pyproject.toml
+src/kattis2canvas/__init__.py
+src/kattis2canvas/__main__.py
+src/kattis2canvas/cli.py
+src/kattis2canvas.egg-info/PKG-INFO
+src/kattis2canvas.egg-info/SOURCES.txt
+src/kattis2canvas.egg-info/dependency_links.txt
+src/kattis2canvas.egg-info/entry_points.txt
+src/kattis2canvas.egg-info/requires.txt
+src/kattis2canvas.egg-info/top_level.txt

kattis2canvas-0.1.0/src/kattis2canvas.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

kattis2canvas-0.1.0/src/kattis2canvas.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+kattis2canvas = kattis2canvas.cli:top

kattis2canvas-0.1.0/src/kattis2canvas.egg-info/requires.txt

@@ -0,0 +1,4 @@
+click>=8.0
+requests>=2.25
+beautifulsoup4>=4.9
+canvasapi>=2.0

kattis2canvas-0.1.0/src/kattis2canvas.egg-info/top_level.txt

@@ -0,0 +1 @@
+kattis2canvas