Automate DVC authentication when using github actions - authentication

Question

I'm using GitHub Actions to run some tests on every push and I need DVC. I'm trying to make this work with the runs-on: ubuntu-latest option but when I try to run it, the action get's stuck because it requires manual authentication. Is there a way to go around this and automate the authentication ?

Answer 1

I figured out that in order for this to work, I had to switch to using a service account.

Answer 2

I had the same issue. Here are the steps I used to setup a Google Service Account with DVC, Google Drive and Github Actions.
Setup a google cloud platform service account
A google cloud platform service account "is a special kind of account used by an application or compute workload, such as a Compute Engine virtual machine (VM) instance, rather than a person. A service account is identified by its email address, which is unique to the account." In this case, the service account for your app to access Google Drive.
Go to the Google API Console
Select or Create a project for DVC remote connections.
Enable the Google Drive API by clicking the ENABLE button
Go to the Google API Crediential page
Click on the + Create Credentials button and then select Service Account.
Enter a service account name (such as gdrive-api) and a some description (such as "Allows access to the google drive api") then click on the Create and continue button.
Click continue at the "Grant this service account access to the project (optional)" section: No need to add any accesses.
Click Done at the "Grant users access to this service account (optional)". Again, no need to grant any additional accesses.
Go to Service Accounts. You'll see a list of all service accounts for the your GCP project and click on the service account you just created.
Click on key --> add key --> Create new key and
In the popup window that appears, select JSON as key type, click the Create button. A JSON file of the key you created will download.
Share access to the Google Drive folder where the DVC repo will live
Still on Google Cloud Platform,
Go in IAM --> Service Accounts and click on the service account name you created in the previous section
Copy and paste the email associated to this service account. It should have a form service_account_name#project_id.iam.gserviceaccount.com
Move to Google Drive and share the drive folder where the DVC repo will live with the service account email defined in the last step.
Set DVC config to use a google service account
On your local machine, modify the DVC project config file by typing the following commands. Change my_remote with your actual DVC remote name.
dvc remote add --default my_remote gdrive://your_folder_hash
dvc remote modify my_remote gdrive_use_service_account true
dvc remote modify my_remote --local gdrive_service_account_json_file_path path/to/file.json
This will modify the .dvc/config and .dvc/config.local files in your project.
Run the command dvc push. If you see no error message, you are done with this step.
Push your code to origin on Github
Setup a Repository Secret
In your Github repo, go to Settings -> Secrets and Variables -> Actions
Click on the New repository secret button.
Name the secret to whatever is meaningful to you.
Copy and paste the JSON file content you downloaded in the previous section to the Secret* text box.
Click the Add Secret button
Setup a Github action
To your Github action yaml file, add the following (make sure to change the YOUR_REPO_SECRET_NAME to the name of the repo secret you created in the previous step.
- uses: iterative/setup-dvc#v1
- name: Pull data with DVC
env:
GDRIVE_CREDENTIALS_DATA: ${{ secrets.YOUR_REPO_SECRET_NAME }}
run: |
dvc pull
Commit this change and run the Github action!
References
How to Setup a Google Drive DVC Remote