I am trying to use kaggle command line tool and I am running into problems with using it inside my own vm. I downloaded the API token from the site and placed it in /.kaggle/kaggle.json on windows. My vm has ubuntu installed and in the Vagrant file I have the following:
config.vm.synced_folder ENV['HOME'] + "/.kaggle", "/home/ubuntu/.kaggle", mount_options: ['dmode=700,fmode=700']
config.vm.provision "shell", inline: <<-SHELL
echo "export KAGGLE_CONFIG_DIR='/home/ubuntu/.kaggle/kaggle.json'" >> /etc/profile.d/myvar.sh
SHELL
when running env command in the vm I see it is correct:
KAGGLE_CONFIG_DIR=/home/ubuntu/.kaggle/kaggle.json
However, when I try to use the kaggle command for example kaggle -h I get the the following
(main) vagrant#dev:/home/ubuntu/.kaggle$ ls
kaggle.json
(main) vagrant#dev:/home/ubuntu/.kaggle$ kaggle -h
Traceback (most recent call last):
File "/user/home/venvs/main/bin/kaggle", line 5, in <module>
from kaggle.cli import main
File "/user/home/venvs/main/lib/python3.7/site-packages/kaggle/__init__.py", line 23, in <module>
api.authenticate()
File "/user/home/venvs/main/lib/python3.7/site-packages/kaggle/api/kaggle_api_extended.py", line 149, in authenticate
self.config_file, self.config_dir))
OSError: Could not find kaggle.json. Make sure it's located in /home/ubuntu/.kaggle/kaggle.json. Or use the environment method.
The paths are all correct and the file is where it should be looking for it. Anyone know what the issue could be? Is it because it is mounted?
Alright, I misread the instructions: "You can define a shell environment variable KAGGLE_CONFIG_DIR to change this location to $KAGGLE_CONFIG_DIR/kaggle.json"
So the env variable should be /home/ubuntu/.kaggle/ instead of /home/ubuntu/.kaggle/kaggle.json.
Related
everyone i'm using oracle arm/amd processor servers with ubuntu 22.04
i have searching for an solution for this over 1 week more than 100 hours
but i had no luck to figure out what is the problem
so if you know about this please help me!
what is problem :
here's a simply code with pyperclip
import pyperclip
string = 'testing'
pyperclip.copy(string)
a = pyperclip.paste()
print()
whenever i execute this code directly in the cli , it is okay to run
However whenever i run this code with schedulers like Crontab or Systemd.timer
(xclip && xsel is already installed)
i got this error and cannot run the code like below
Traceback (most recent call last):
File "/home/ubuntu/web_selenium/pytest.py", line 10, in <module>
pyperclip.copy(string)
File "/usr/lib/python3/dist-packages/pyperclip/__init__.py", line 689, in lazy_load_stub_copy
return copy(text)
File "/usr/lib/python3/dist-packages/pyperclip/__init__.py", line 358, in __call__
raise PyperclipException(EXCEPT_MSG)
pyperclip.PyperclipException:
Pyperclip could not find a copy/paste mechanism for your system.
For more information, please visit https://pyperclip.readthedocs.io/en/latest/index.html#not-implemented-error
i tried most of solutions in stackoverflow, mostly it was about setting path for interpreter and packages
of course i installed xclip and xsel as well / everything is good when i run code directly through cli
but only when i run code or shell script through Crontab , Systemd.timer, this error is generated
what i have tried
1. run pyperclip code through shell script with various python env PATH
i tried to lay all the possible env path like /usr/bin:/usr/local/bin .......
and even my python pakages path as .profile and ~./bashrc - failed
2. set new venv and install all the modules, run code - failed
3. run crontab as root - failed
4. install new ubuntu and switch server ubuntu amd -> ubuntu aarch64 - failed
5. reinstall module - failed
6. set shebang inside of python script ( whereis python3 - inside of python file - #!/usr/bin/python3.10) - failed
7. set shell into crontab (SHELL=/bin/bash) - failed
now i'm thinking this is the problem of pyperclip
pleaese help me!
I have an scp function as follows:
def scp_to38():
child = pexpect.spawn(f"scp {tar_file_path} bbmprd#10.82.20.38:/app2/upload/")
child.expect(f"bbmprd#10.82.20.38's password:")
child.sendline("password")
child.interact()
print(f'{tar_file_path} SENT TO 38:/app2/upload/')
and it is throwing me an error saying a problem with child.interact()
Traceback (most recent call last):
File "scan_for_upload.py", line 39, in <module>
scp_to38()
File "scan_for_upload.py", line 26, in scp_to38
child.interact()
File "/app/anaconda3/envs/python37-1/lib/python3.7/site-packages/pexpect/pty_spawn.py", line 788, in interact
mode = tty.tcgetattr(self.STDIN_FILENO)
termios.error: (25, 'Inappropriate ioctl for device')
I am running the script in a nohup process, I believe that is causing this issue. Because when I run the script in the console, it works fine. How do I interact with the scp password in a nohup process using pexpect module? (I can't install any other external modules in my environment)
interact() is for manual interaction with the spawned process and it requires to be running on a tty but nohup provides no tty. You can just replace child.interact() with child.expect(pexpect.EOF, timeout=None).
I've recently learned the basics of Docker and how to create and run images. I'm trying to create an image of a python script that scrapes some webpages for data and uploads it to a server. I'm using Selenium, Chromium, and a Windows chromedriver. I'm trying to build the image on my Windows machine and be able to deploy it on a bunch of Linux/Windows servers. Currently, I'm only building and running on the same Windows machine, just until I get it running, but I keep getting the same error, even though the script runs just fine directly on the machine itself.
This is the error:
Traceback (most recent call last):
File "my-app.py", line 796, in <module>
startScraper();
File "my-app.py", line 92, in startScraper
browser = webdriver.Chrome(chrome_options = options, executable_path = path_to_chromedriver);
File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 62, in __init__
self.service.start()
File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 74, in start
stdout=self.log_file, stderr=self.log_file)
File "/usr/local/lib/python3.6/subprocess.py", line 707, in __init__
restore_signals, start_new_session)
File "/usr/local/lib/python3.6/subprocess.py", line 1326, in _execute_child
raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error
It seems to be related to the chrome options but even when I remove all the "add-argument" options, the error persists, but here are the options:
options = webdriver.ChromeOptions();
options.binary_location = './chrome-win32/chrome.exe';
options.add_argument('headless')
options.add_argument('window-size=1400x1300')
options.add_argument('--mute-audio')
options.add_argument('--disable-web-security');
options.add_argument('--allow-running-insecure-content');
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
prefs = {"profile.managed_default_content_settings.images":2}
options.add_experimental_option("prefs", prefs);
path_to_chromedriver = './chromedriver.exe';
Is there anything that I'm missing to be able to run this scraper in a container? Thanks!
EDIT: I forgot to add the Dockerfile and how I build/run the image:
Dockerfile:
FROM python:3.6.0
WORKDIR /my-app
ADD . /my-app
RUN pip install -r requirements.txt
ENV NAME Scraper
CMD ["python", "My_App.py"]
Build/Run image:
- docker build -t myapp
- docker run myapp
Maybe there are some options that I don't know about that I'm missing?
You are trying to run a exe inside a linux container and that is not going to work. You will need to install chrome and chromedriver inside your Dockerfile and update the code to use the correct path
FROM python:3.6.0
RUN apt update && apt install -y chromedriver
WORKDIR /my-app
ADD . /my-app
RUN pip install -r requirements.txt
ENV NAME Scraper
CMD ["python", "My_App.py"]
Change your code to
options = webdriver.ChromeOptions();
options.add_argument('headless')
options.add_argument('window-size=1400x1300')
options.add_argument('--mute-audio')
options.add_argument('--disable-web-security');
options.add_argument('--allow-running-insecure-content');
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
prefs = {"profile.managed_default_content_settings.images":2}
options.add_experimental_option("prefs", prefs);
path_to_chromedriver = '/usr/lib/chromium/chromedriver';
I'm trying to connect to a remote rabbitmq host using the cli rabbitmqadmin.
The command I'm trying to execute is:
rabbitmqadmin --host=$RABBITMQ_HOST --port=443 --ssl --vhost=$RABBITMQ_VHOST --username=$RABBITMQ_USERNAME --password=$RABBITMQ_PASSWORD list queues
Before you ask: the environmental variables RABBITMQ_HOST, RABBITMQ_VHOST and so on are set... I double and triple checked this already.
The error I get back is:
Traceback (most recent call last):
File "/usr/local/sbin/rabbitmqadmin", line 1007, in <module>
main()
File "/usr/local/sbin/rabbitmqadmin", line 413, in main
method()
File "/usr/local/sbin/rabbitmqadmin", line 588, in invoke_list
format_list(self.get(uri), cols, obj_info, self.options)
File "/usr/local/sbin/rabbitmqadmin", line 436, in get
return self.http("GET", "%s/api%s" % (self.options.path_prefix, path), "")
File "/usr/local/sbin/rabbitmqadmin", line 475, in http
self.options.port)
File "/usr/local/sbin/rabbitmqadmin", line 451, in __initialize_https_connection
context = self.__initialize_tls_context())
File "/usr/local/sbin/rabbitmqadmin", line 467, in __initialize_tls_context
self.options.ssl_key_file)
TypeError: coercing to Unicode: need string or buffer, NoneType found
From the last line I assume it's a python related problem, my current python version is 2.7.12, if I try to connect to the local instance of rabbitmq with
rabbitmqadmin list queues
everything works fine. Any help is greatly appreciated thanks :)
shouldn't those env vars have a $ in front of them, and the params without =?
rabbitmqadmin --host $RABBITMQ_HOST --port 443 --ssl --vhost $RABBITMQ_VHOST --username $RABBITMQ_USERNAME --password $RABBITMQ_PASSWORD list queues`
maybe the = doesn't matter, but i'm pretty sure you need $ in front of the env vars
Validate that you are using the same rabbitmqadmin version as the version of your remote hosted broker. Using a mismatching rabbitmqadmin version will result in that error (for example rabbitmqadmin 3.6.4 querying a 3.5.7 server).
Browse to http://server-name:15672/cli/ and download correct tool from there.
https://github.com/rabbitmq/rabbitmq-management/issues/299
I have a self-hosted gitlab and I would like to install a package hosted there using ssh.
I tried:
pip install git+ssh://git#<my_domain>:se7entyse7en/<project_name>.git
Here's the output:
Downloading/unpacking git+ssh://git#<my_domain>:se7entyse7en/<project_name>.git
Cloning ssh://git#<my_domain>:se7entyse7en/<project_name>.git to /var/folders/3r/v7swlvdn2p7_wyh9wj90td2m0000gn/T/pip-4_JdRU-build
ssh: Could not resolve hostname <my_domain>:se7entyse7en: nodename nor servname provided, or not known
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
Update:
I tried to upload it on gitlab.com and after having uploaded the repo I tried to install it by running:
pip install git+ssh://git#gitlab.com:loumarvincaraig/<project_name>.git
but the nothing changed. In particular here's the content of pip.log:
/Users/se7entyse7en/Envs/test/bin/pip run on Mon Nov 17 22:14:51 2014
Downloading/unpacking git+ssh://git#gitlab.com:loumarvincaraig/<project_name>.git
Cloning ssh://git#gitlab.com:loumarvincaraig/<project_name>.git to /var/folders/3r/v7swlvdn2p7_wyh9wj90td2m0000gn/T/pip-91JVFi-build
Found command 'git' at '/usr/local/bin/git'
Running command /usr/local/bin/git clone -q ssh://git#gitlab.com:loumarvincaraig/<project_name>.git /var/folders/3r/v7swlvdn2p7_wyh9wj90td2m0000gn/T/pip-91JVFi-build
Complete output from command /usr/local/bin/git clone -q ssh://git#gitlab.com:loumarvincaraig/<project_name>.git /var/folders/3r/v7swlvdn2p7_wyh9wj90td2m0000gn/T/pip-91JVFi-build:
Cleaning up...
Command /usr/local/bin/git clone -q ssh://git#gitlab.com:loumarvincaraig/<project_name>.git /var/folders/3r/v7swlvdn2p7_wyh9wj90td2m0000gn/T/pip-91JVFi-build failed with error code 128 in None
Exception information:
Traceback (most recent call last):
File "/Users/se7entyse7en/Envs/test/lib/python2.7/site-packages/pip/basecommand.py", line 134, in main
status = self.run(options, args)
File "/Users/se7entyse7en/Envs/test/lib/python2.7/site-packages/pip/commands/install.py", line 236, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/Users/se7entyse7en/Envs/test/lib/python2.7/site-packages/pip/req.py", line 1092, in prepare_files
self.unpack_url(url, location, self.is_download)
File "/Users/se7entyse7en/Envs/test/lib/python2.7/site-packages/pip/req.py", line 1231, in unpack_url
return unpack_vcs_link(link, loc, only_download)
File "/Users/se7entyse7en/Envs/test/lib/python2.7/site-packages/pip/download.py", line 410, in unpack_vcs_link
vcs_backend.unpack(location)
File "/Users/se7entyse7en/Envs/test/lib/python2.7/site-packages/pip/vcs/__init__.py", line 240, in unpack
self.obtain(location)
File "/Users/se7entyse7en/Envs/test/lib/python2.7/site-packages/pip/vcs/git.py", line 111, in obtain
call_subprocess([self.cmd, 'clone', '-q', url, dest])
File "/Users/se7entyse7en/Envs/test/lib/python2.7/site-packages/pip/util.py", line 670, in call_subprocess
% (command_desc, proc.returncode, cwd))
InstallationError: Command /usr/local/bin/git clone -q ssh://git#gitlab.com:loumarvincaraig/<project_name>.git /var/folders/3r/v7swlvdn2p7_wyh9wj90td2m0000gn/T/pip-91JVFi-build failed with error code 128 in None
I don't know why, but by running the following command it worked (slash instead of : after <my_domain>):
pip install git+ssh://git#<my_domain>/se7entyse7en/<project_name>.git
# ^
# slash instead of :
Yes. This is the default use:
pip install git+ssh://git#<my_domain>:22/<project_group>/<project_name>.git
The use of the colon by itself implies the default ssh port number 22. Because you can control the port number of your server, the port number could be different. Git enables customisation by not providing :22/ or / only.