Skip to content
Snippets Groups Projects
Commit 633bc756 authored by Jan Lukas Steimann's avatar Jan Lukas Steimann
Browse files

Add S3 config and use real csv file

parent c4166b64
Branches
No related tags found
No related merge requests found
Pipeline #64544 failed
image: "python:3.7" image: "python:3.7"
before_script:
- wget https://dl.min.io/client/mc/release/linux-amd64/mc
- chmod +x mc
- mc alias set S3Uni $MINIO_HOST $MINIO_ACCESS_KEY $MINIO_SECRET_KEY
python_run: python_run:
script: script:
- pip install flair - pip install flair
- pip install torch - pip install torch
- pip install pandas - pip install pandas
- mc cp S3Uni/steimann/CommentsApril2017.csv .
- python3 ComputeCommentEmbeddings.py - python3 ComputeCommentEmbeddings.py
- mc cp embedded_comments.pt S3Uni/steimann/
tags: tags:
- "ht" - "ht"
\ No newline at end of file
...@@ -18,6 +18,6 @@ def compute_embedding(comments: dict) -> dict: ...@@ -18,6 +18,6 @@ def compute_embedding(comments: dict) -> dict:
if __name__ == '__main__': if __name__ == '__main__':
comments = get_argument_list(path="", filename="CommentsApril2017_DEV.csv") comments = get_argument_list(path="", filename="CommentsApril2017.csv")
comments = compute_embedding(comments) comments = compute_embedding(comments)
torch.save(comments, "embedded_comments.pt") torch.save(comments, "embedded_comments.pt")
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment