Submitting results to Kaggle competition from command line regardless of kernel type or file name, in or out of Kaggle - google-colaboratory

Question

Within Kaggle: How can I submit my results to Kaggle competition regardless of kernel type or file name?
And if I am in a notebook outside Kaggle (Colab, Jupyter, Paperspace, etc.)?

Answer 1

Introduction (you can skip this part)
I was looking around for a method to do that. In particular, being able to submit at any point within the notebook (so you can test different approaches), a file with any name (to keep things separated), and any number of times (respecting the Kaggle limitations).
I found many webs explaining the process like
Making Submission
1. Hit the "Publish" button at the top of your notebook screen.
If you have written an output file, then you have an "Output" tab.
2. Output > Submit to Competition
However they fail to clarify that the Kernel must be of type "Script" and not "Notebook".
That has some limitations that I haven't fully explored.
I just wanted to be able to submit whatever file from the notebook, just like any other command within it.
The process
Well, here is the process I came up with.
Suggestions, errors, comments, improvements are welcome. Specifically I'd like to know why this method is no better than the one described above.
Process:
Install required libraries
Provide your kaggle credentials
using the file kaggle.json OR
setting some environment variables with your kaggle credentials
Submit with a simple command.
Q: Where do I get my kaggle credentials?
A: You get them from https://www.kaggle.com > 'Account' > "Create new API token"
1. Install required libraries
# Install required libraries
!pip install --upgrade pip
!pip install kaggle --upgrade
2. Provide your kaggle credentials -- setting some environment variables with your kaggle credentials
# Add your PRIVATE credentials
# Do not use "!export KAGGLE_USERNAME= ..." OR "" around your credential
%env KAGGLE_USERNAME=abc
%env KAGGLE_KEY=12341341
# Verify
!export -p | grep KAGGLE_USERNAME
!export -p | grep KAGGLE_KEY
See Note below.
2. Provide your kaggle credentials -- using the file kaggle.json
%mkdir --parents /root/.kaggle/
%cp /kaggle/input/<your_private_dataset>/kaggle.json /root/.kaggle/
!chmod 600 /root/.kaggle/kaggle.json
How you get the file there is up to you.
One simple way is this:
Download the kaggle.json to your computer
In kaggle, create a private dataset (Your_Profile > Datasets > New Dataset)
Add the kaggle.json to that Dataset
Add the private Dataset to your notebook ( Data > Add Data > Datasets > Your Datasets)
This may seem a bit cumbersome, but soon or later your API credentials may change and updating the file in one point (the dataset) will update it in all your notebooks.
(source: googleapis.com)
3. Submit with a simple command.
Here <competition-name> is the code name of the competition. You can get it from the url of the competition or from the section "My submissions" within the competition page.
(source: googleapis.com)
# Submit
!kaggle competitions submit -c <competition-name> -f submission.csv -m "Notes"
# example:
!kaggle competitions submit -c bike-sharing-demand -f submission.csv -m "Notes"
# View results
!kaggle competitions submissions -c <competition-name>
# example:
!kaggle competitions submissions -c bike-sharing-demand
Note:
If you are too conscious about security of your credentials and/or want to share the kernel, then you can type the 2 commands with your credentials on the "Console" instead of within the notebook (example below). They will be valid/available during that session only.
import os
os.environ['KAGGLE_USERNAME'] = "here DO use double quotes"
os.environ['KAGGLE_KEY'] = "here DO use double quotes"
You can find the console at the bottom of your kernel.
(source: googleapis.com)
PS: Initially this was posted here, but when the answer grew the Markdown display breaks in Kaggle (not in other places), therefore I had to take it out of Kaggle.