I had a Windows 10 machine working great with bq/gloud and gsutils all working great. Had to move to new machine and setting all of this back up. I cannot seem to get bq to recognize the entries in my .bigqueryrc file which was copied from my working machine. On my working machine, I am not using BIGQUERYRC environmental file, but just HOME, ie ~/.bigqueryrc
I say it does not seem to recognize is because if I introduce a random character I get an error about my .bigqueryrc file.
Let me first show that bq seems to be operating, overall, as expected
C:\Users\boyer>bq ls --project_id=broad-tapestry-sbx-boyer-14
datasetId
---------------
Broad_DataGov
Broad_EDW
Broad_Kitchen
Broad_Lake
Broad_Tableau
Broad_Utils
SHELL> bq ls brings nothing. The contents of my .biqqueryrc file is as thus:
credential_file = C:\Users\boyer\AppData\Roaming\gcloud\legacy_credentials\boyer#broadinstitute.org\singlestore_bq.json
project_id = broad-tapestry-sbx-boyer-14
[query]
--use_legacy_sql=false
Here is showing where my .bigqueryrc file is:
:\Users\boyer>ls -ls ~/.bigqueryrc
1 -rwx------+ 1 CHARLES+boyer CHARLES+boyer 198 Dec 22 12:48 /cygdrive/c/Users/boyer/.bigqueryrc
So, I try this and nothing happens either:
C:\Users\boyer>bq ls --bigqueryrc=c:/users/boyer/.bigqueryrc
C:\Users\boyer>bq ls --bigqueryrc=c:\users\boyer\.bigqueryrc
C:\Users\boyer>
But here is the error I purposefully introduced to see if it is actually looking at the file and it seems it is finding it.
C:\Users\boyer>bq ls --bigqueryrc=c:\users\boyer\.bigqueryrc
Unknown flag x found in bigqueryrc file in section global
The error above was introduced by adding a random 'x' character to the top of the file:
x
credential_file = C:\Users\boyer\AppData\Roaming\gcloud\legacy_credentials\boyer#broadinstitute.org\singlestore_bq.json
project_id = broad-tapestry-sbx-boyer-14
[query]
--use_legacy_sql=false
Here's my .vimrc
call plug#begin('~/.vim/plugged')
Plug 'dart-lang/dart-vim-plugin'
Plug 'natebosch/vim-lsc'
Plug 'natebosch/vim-lsc-dart'
Plug 'tpope/vim-sensible'
Plug 'vim-airline/vim-airline'
Plug 'w0rp/ale'
Plug 'pearofducks/ansible-vim', { 'do': './UltiSnips/generate.sh' }
Plug 'ncm2/ncm'
Plug 'roxma/nvim-yarp'
Plug 'ncm2/ncm2-bufword'
Plug 'ncm2/ncm2-path'
Plug 'iamcco/markdown-preview.nvim', { 'do': 'cd app && yarn install' }
Plug 'codota/tabnine-vim'
Plug 'Chiel92/vim-autoformat'
Plug 'neoclide/coc.nvim', {'branch': 'release'}
Plug 'jiangmiao/auto-pairs'
Plug 'autozimu/LanguageClient-neovim', {
\ 'branch': 'next',
\ 'do': 'bash install.sh',
\}
" Plug 'valloric/youcompleteme'
let g:lsc_server_commands = {'dart': 'dart_language_server'}
call plug#end()
let g:ale_fixers = {
\ 'javascript': ['eslint']
\}
let g:ale_sign_error = '❌'
let g:ale_sign_warning = '⚠️'
let g:lsc_enable_autocomplete = v:true
let g:lsc_auto_map = v:true
let g:ale_fix_on_save = 1
let g:ale_javascript_prettier_use_local_config = 1
let g:ale_javascript_eslint_options = '-c ~/.eslintrc'
let g:ale_lint_on_enter = 0
let g:ansible_unindent_after_newline = 1
autocmd BufEnter * call ncm2#enable_for_buffer()
set completeopt=noinsert,menuone,noselect
" set softtabstop=2
" set shiftwidth=2
" set expandtab
set expandtab
" show existing tab with 2 spaces width
set tabstop=2
set softtabstop=2
" when indenting with '>', use 2 spaces width
set shiftwidth=2
nnoremap <A-Down> :m .+1<CR>==
nnoremap <A-Up> :m .-2<CR>==
inoremap <A-Down> <Esc>:m .+1<CR>==gi
inoremap <A-Up> <Esc>:m .-2<CR>==gi
vnoremap <A-Down> :m '>+1<CR>gv=gv
vnoremap <A-Up> :m '<-2<CR>gv=gv
set statusline=%<%f\ %h%m%r%{kite#statusline()}%=%-14.(%l,%c%V%)\ %P
set laststatus=2
let g:python3_host_prog='/home/thebozzkg/anaconda3/bin/python3'
au BufWrite * :Autoformat
let g:airline#extensions#tabline#enabled = 1
let g:airline_detect_spell=1
let g:airline_left_sep = ''
let g:airline_left_alt_sep = ''
let g:airline_right_sep = ''
let g:airline_right_alt_sep = ''
let g:airline_symbols.branch = ''
let g:airline_symbols.readonly = ''
let g:airline_symbols.linenr = '☰'
let g:airline_symbols.maxlinenr = ''
let g:airline_symbols.dirty='⚡'
nnoremap <ALT-Left> :tabprevious<CR>
nnoremap <ALT-Right> :tabnext<CR>
Some notes about me:
I don't use hjkl I use the arrow keys (pls don't insult me)
I write Javascript, Dart, Markdown, YAML, JSON, C, & C++ (so far) in neovim (I'm switching from VSCode)
Most of the stuff I have I literally copied from GitHub and some Vim wikis
I have Anaconda installed
I'm using Ubuntu Linux 20.04 (if that matters)
For some reason, I don't know which plugins are doing it but my autocompletion keeps messing up my files (editing YAML files and saving them are absolute torture, some indentation is lost with almost all files) and also crashing (TabNine)
Can someone help me with this mess?
Welcome to Vim!
I think most new vim users have been there. I certainly have! I wanted a
'vim as python IDE' and copied a whole bunch of stuff from every blog under the
sun into my vimrc almost immediately after installing vim.
After some time spent fighting with all the settings, plugins and remaps I
didn't understand, I decided to go through my vimrc, line by line and comment
out anything I didn't understand (nearly all of it).
Then I used this more minimal vim for a while and whenever I decided I had a
need for a certain feature, I checked the largely commented vimrc for anything
that looked related, and/or googled for that particular feature only. Often
you find that there is a built in method to do it with the core vim commands,
and if not, then there are a lot of solutions for the problem (and often, you
find that there is an even more powerful way that didn't occur to you - these
are good days).
But the key is to not try and coerce vim into a huge IDE overnight! Let it
happen gradually and things will make more sense, and you'll end up with a
vimrc that you understand and therefore be in a position to add to it and tweak
it.
The last thing I'll say is to recommend the following books:
'Learn VimScript the Hard Way' by Steve Losh
'Practical Vim' by Drew Neil, and his accompanying screencast series.
(Also there is The Primagean who does high quality youtube tutorials)
Having said all that, and acknowledging that an objective answer can't be given
for your question, here is a minimal vimrc which has a few plugins and settings
that do simple but very useful things (but do read up on them to understand how
they work!):
"==== PLUGINS =================================================================
set nocompatible " don't try to be compatible with Vi
filetype plugin indent on " use default plugins
call plug#begin('~/.vim/plugged')
" plugins I would put in a new vimrc
Plug 'tpope/vim-sensible'
Plug 'tpope/vim-repeat'
Plug 'tpope/vim-surround'
Plug 'tpope/vim-commentary'
Plug 'vim-scripts/ReplaceWithRegister'
Plug 'flazz/vim-colorschemes'
"==== CUSTOM CONFIGURATIONS ===================================================
"- general settings -----------------------------------------------------------
set encoding=utf-8
set linebreak " wrap long lines at char in 'breakat' (default " ^I!#*-+;:,./?")
set nowrap " don't wrap lines by default
set expandtab " expand tabs into spaces
set tabstop=4 " a tab is the same as 4 spaces
set softtabstop=4 " when I hit <tab> in insert mode, put 4 spaces
set shiftwidth=4 " when auto-indenting, use 4 spaces per tab
set autoindent " when creating a new line, copy indent from line above
set incsearch " show matches for patterns while they are being typed
set smartcase " with both on, searches with no capitals are case insensitive...
set ignorecase " ...while searches with capital characters are case sensitive.
set spell spelllang=en
set nospell " don't hightlight misspellings unles I say so
set lazyredraw " don't redraw screen during macros (let them complete faster)
set t_Co=256 " use full colours
syntax enable " highlight special words to aid readability
colorscheme zenburn
" THIS NEXT BIT I DEBATED INCLUDING - IT'S MY PERSONAL PREFERENCE AND BY
NO MEANS 'NECESSARY' AND COULD SAFELY BE EXCLUDED... BUT I JUST HATED THE
BLOCK CURSOR IN INSERT MODE SO MUCH...
"- cursor behaviour (make cursor blink for insert mode, block for normal mode)
augroup cursor_behaviour
autocmd!
" reset cursor on start:
autocmd VimEnter * silent !echo -ne "\e[2 q"
" cursor blinking bar on insert mode
let &t_SI = "\e[5 q"
" cursor steady block on command mode
let &t_EI = "\e[2 q"
" highlight current line when in insert mode
autocmd InsertEnter * set cursorline
" turn off current line hightlighting when leaving insert mode
autocmd InsertLeave * set nocursorline
augroup END
Good luck!
I wrote the below code, which will extract the directory name along with the file name and I will use purge command on that extracted Text.
$ sear VAXMANAGERS_ROOT:[PROC]TEMP.LIS LOG/out=VAXMANAGERS_ROOT:[DEV]FVLIM.TXT
$ OPEN IN VAXMANAGERS_ROOT:[DEV]FVLIM.TXT
$ LOOP:
$ READ/END_OF_FILE=ENDIT IN ABCD
$ GOTO LOOP
$ ENDIT:
$ close in
$ ERROR=F$EXTRACT(0,59,ABCD)
$ sh sym ERROR
$ purge/keep=1 'ERROR'
The output is as follows:
ERROR = "$1$DKC102:[PROD_LIVE.LOG]DP2017_TMP2.LIS;27392 "
Problem here is --- Every time the directory length varies (Length may be 59 or 40 or some other value, but the directory and filename length will not exceed 59 characters in my system). So in the above output, the system is also fetching the Version number of that file number. So I am not able to purge the file along with the version number.
%PURGE-E-PURGEVER, version numbers not permitted
Any suggestion -- How to eliminate the version number from the output ?
I cannot use the exact length of the directory, as directory length varies everytime.... :(
The answer with F$ELEMENT( 0, ";", ABCD ) should work, as confirmed. I might script something like this:
$ ERROR = F$PARSE(";",ERROR) ! will return $1$DKC102:[PROD_LIVE.LOG]DP2017_TMP2.LIS;
$ ERROR = ERROR - ";"
$ PURGE/KEEP=1 'ERROR'
Not sure why you have the read loop. What you will get is the last line in the file, but assuming that's what you want.
While HABO explained it, some more explanations
Suppose I use f$search to check if a file exists
a = f$search("sys$manager:net$server.log")
then I find I it exists
wr sys$output a
shows
SYS$SYSROOT:[SYSMGR]NET$SERVER.LOG;9
From the help of f$parse I get
help lex f$parse arg
shows, among other things
`Specifies a character string containing the name of a field
in a file specification. Specifying the field argument causes
the F$PARSE function to return a specific portion of a file
specification.
Specify one of the following field names (do not abbreviate):
NODE Node name
DEVICE Device name
DIRECTORY Directory name
NAME File name
TYPE File type
VERSION File version number`
So I can do
wr sys$output f$parse(a,,,"DEVICE")
which shows
SYS$SYSROOT:
and also
wr sys$output f$parse(a,,,"DIRECTORY")
so I get
[SYSMGR]
and
wr sys$output f$parse(a,,,"NAME")
shows
NET$SERVER
and
wr sys$output f$parse(a,,,"TYPE")
shows
.LOG
the version is
wr sys$output f$parse(a,,,"VERSION")
shown as
;9
The lexicals functions can be handy, check it using
help lexical
it shows
F$CONTEXT F$CSID F$CUNITS F$CVSI F$CVTIME F$CVUI F$DELTA_TIME F$DEVICE F$DIRECTORY F$EDIT
F$ELEMENT F$ENVIRONMENT F$EXTRACT F$FAO F$FID_TO_NAME F$FILE_ATTRIBUTES F$GETDVI F$GETENV
F$GETJPI F$GETQUI F$GETSYI F$IDENTIFIER F$INTEGER F$LENGTH F$LICENSE F$LOCATE F$MATCH_WILD
F$MESSAGE F$MODE F$MULTIPATH F$PARSE F$PID F$PRIVILEGE F$PROCESS F$READLINK F$SEARCH
F$SETPRV F$STRING F$SYMLINK_ATTRIBUTES F$TIME F$TRNLNM F$TYPE F$UNIQUE F$USER
How do I get gvim to recognize sqlcomplete.vim commands?
I'm unable to use the sqlcomplete.vim plugin. When running :version I get the following output:
and scrolling all the way to the bottom here is the rest of the output:
and the env variables:
:echo $VIM
c:\users\me\.babun\cygwin\etc\
:echo $HOME
H:\
Here is the output of :scriptnames:
When running the sqlcomplete.vim command such as :SQLSetType sqlanywhere the output I get is:
How do I get gvim to recognize sqlcomplete.vim commands?
Another piece of helpful information is the output of :echo &rtp :
H:\vimfiles,H:\.vim\bundle\Vundle.vim,H:\.vim\bundle\dbext.vim,H:\.vim\bundle\SQ
LComplete.vim,C:\Users\me\.babun\cygwin\etc\vimfiles,C:\Users\me\.babu
n\cygwin\etc\,C:\Users\me\.babun\cygwin\etc\vimfiles/after,H:\vimfiles/afte
r,H:\.vim/bundle/Vundle.vim,H:\.vim\bundle\Vundle.vim/after,H:\.vim\bundle\dbext
.vim/after,H:\.vim\bundle\SQLComplete.vim/after
Some points you could check:
:scriptnames shows plugin\sqlcomplete.vim
But the link you provided points to .../vim/runtime/autoload/sqlcomplete.vim, there is no .../vim/runtime/plugin/sqlcomplete.vim, and the version at vim.org also doesn't contains a /plugin file:
install details
Copy sqlcomplete.vim to:
.vim/autoload/sqlcomplete.vim (Unix)
vimfiles\autoload\sqlcomplete.vim (Windows)
For documentation:
:h sql.txt
Maybe you have installed it incorrectly.
The file on your link has version 12 at its header, while the latest version is 15. Try updating to the latest version
Note that this plugin does not define the SQLSetType command.
You could check that by simple searching the file the on link. And on its header:
" Vim OMNI completion script for SQL
" Language: SQL
" Maintainer: David Fishburn <dfishburn dot vim at gmail dot com>
" Version: 15.0
" Last Change: 2013 May 13
" Homepage: http://www.vim.org/scripts/script.php?script_id=1572
" Usage: For detailed help
" ":help sql.txt"
" or ":help ft-sql-omni"
" or read $VIMRUNTIME/doc/sql.txt
Following :help sql.txt:
2.1 SQLSetType *sqlsettype* *SQLSetType*
--------------
For the people that work with many different databases, it is nice to be
able to flip between the various vendors rules (indent, syntax) on a per
buffer basis, at any time. The ftplugin/sql.vim file defines this function: >
SQLSetType
And scriptnames is not listing ftplugin/sql.vim
It's the first great virtue of programmers. All of us have, at one time or another automated a task with a bit of throw-away code. Sometimes it takes a couple seconds tapping out a one-liner, sometimes we spend an exorbitant amount of time automating away a two-second task and then never use it again.
What tiny hack have you found useful enough to reuse? To make go so far as to make an alias for?
Note: before answering, please check to make sure it's not already on favourite command-line tricks using BASH or perl/ruby one-liner questions.
i found this on dotfiles.org just today. it's very simple, but clever. i felt stupid for not having thought of it myself.
###
### Handy Extract Program
###
extract () {
if [ -f $1 ] ; then
case $1 in
*.tar.bz2) tar xvjf $1 ;;
*.tar.gz) tar xvzf $1 ;;
*.bz2) bunzip2 $1 ;;
*.rar) unrar x $1 ;;
*.gz) gunzip $1 ;;
*.tar) tar xvf $1 ;;
*.tbz2) tar xvjf $1 ;;
*.tgz) tar xvzf $1 ;;
*.zip) unzip $1 ;;
*.Z) uncompress $1 ;;
*.7z) 7z x $1 ;;
*) echo "'$1' cannot be extracted via >extract<" ;;
esac
else
echo "'$1' is not a valid file"
fi
}
Here's a filter that puts commas in the middle of any large numbers in standard input.
$ cat ~/bin/comma
#!/usr/bin/perl -p
s/(\d{4,})/commify($1)/ge;
sub commify {
local $_ = shift;
1 while s/^([ -+]?\d+)(\d{3})/$1,$2/;
return $_;
}
I usually wind up using it for long output lists of big numbers, and I tire of counting decimal places. Now instead of seeing
-rw-r--r-- 1 alester alester 2244487404 Oct 6 15:38 listdetail.sql
I can run that as ls -l | comma and see
-rw-r--r-- 1 alester alester 2,244,487,404 Oct 6 15:38 listdetail.sql
This script saved my career!
Quite a few years ago, i was working remotely on a client database. I updated a shipment to change its status. But I forgot the where clause.
I'll never forget the feeling in the pit of my stomach when I saw (6834 rows affected). I basically spent the entire night going through event logs and figuring out the proper status on all those shipments. Crap!
So I wrote a script (originally in awk) that would start a transaction for any updates, and check the rows affected before committing. This prevented any surprises.
So now I never do updates from command line without going through a script like this. Here it is (now in Python):
import sys
import subprocess as sp
pgm = "isql"
if len(sys.argv) == 1:
print "Usage: \nsql sql-string [rows-affected]"
sys.exit()
sql_str = sys.argv[1].upper()
max_rows_affected = 3
if len(sys.argv) > 2:
max_rows_affected = int(sys.argv[2])
if sql_str.startswith("UPDATE"):
sql_str = "BEGIN TRANSACTION\\n" + sql_str
p1 = sp.Popen([pgm, sql_str],stdout=sp.PIPE,
shell=True)
(stdout, stderr) = p1.communicate()
print stdout
# example -> (33 rows affected)
affected = stdout.splitlines()[-1]
affected = affected.split()[0].lstrip('(')
num_affected = int(affected)
if num_affected > max_rows_affected:
print "WARNING! ", num_affected,"rows were affected, rolling back..."
sql_str = "ROLLBACK TRANSACTION"
ret_code = sp.call([pgm, sql_str], shell=True)
else:
sql_str = "COMMIT TRANSACTION"
ret_code = sp.call([pgm, sql_str], shell=True)
else:
ret_code = sp.call([pgm, sql_str], shell=True)
I use this script under assorted linuxes to check whether a directory copy between machines (or to CD/DVD) worked or whether copying (e.g. ext3 utf8 filenames -> fusebl
k) has mangled special characters in the filenames.
#!/bin/bash
## dsum Do checksums recursively over a directory.
## Typical usage: dsum <directory> > outfile
export LC_ALL=C # Optional - use sort order across different locales
if [ $# != 1 ]; then echo "Usage: ${0/*\//} <directory>" 1>&2; exit; fi
cd $1 1>&2 || exit
#findargs=-follow # Uncomment to follow symbolic links
find . $findargs -type f | sort | xargs -d'\n' cksum
Sorry, don't have the exact code handy, but I coded a regular expression for searching source code in VS.Net that allowed me to search anything not in comments. It came in very useful in a particular project I was working on, where people insisted that commenting out code was good practice, in case you wanted to go back and see what the code used to do.
I have two ruby scripts that I modify regularly to download all of various webcomics. Extremely handy! Note: They require wget, so probably linux. Note2: read these before you try them, they need a little bit of modification for each site.
Date based downloader:
#!/usr/bin/ruby -w
Day = 60 * 60 * 24
Fromat = "hjlsdahjsd/comics/st%Y%m%d.gif"
t = Time.local(2005, 2, 5)
MWF = [1,3,5]
until t == Time.local(2007, 7, 9)
if MWF.include? t.wday
`wget #{t.strftime(Fromat)}`
sleep 3
end
t += Day
end
Or you can use the number based one:
#!/usr/bin/ruby -w
Fromat = "http://fdsafdsa/comics/%08d.gif"
1.upto(986) do |i|
`wget #{sprintf(Fromat, i)}`
sleep 1
end
Instead of having to repeatedly open files in SQL Query Analyser and run them, I found the syntax needed to make a batch file, and could then run 100 at once. Oh the sweet sweet joy! I've used this ever since.
isqlw -S servername -d dbname -E -i F:\blah\whatever.sql -o F:\results.txt
This goes back to my COBOL days but I had two generic COBOL programs, one batch and one online (mainframe folks will know what these are). They were shells of a program that could take any set of parameters and/or files and be run, batch or executed in an IMS test region. I had them set up so that depending on the parameters I could access files, databases(DB2 or IMS DB) and or just manipulate working storage or whatever.
It was great because I could test that date function without guessing or test why there was truncation or why there was a database ABEND. The programs grew in size as time went on to include all sorts of tests and become a staple of the development group. Everyone knew where the code resided and included them in their unit testing as well. Those programs got so large (most of the code were commented out tests) and it was all contributed by people through the years. They saved so much time and settled so many disagreements!
I coded a Perl script to map dependencies, without going into an endless loop, For a legacy C program I inherited .... that also had a diamond dependency problem.
I wrote small program that e-mailed me when I received e-mails from friends, on an rarely used e-mail account.
I wrote another small program that sent me text messages if my home IP changes.
To name a few.
Years ago I built a suite of applications on a custom web application platform in PERL.
One cool feature was to convert SQL query strings into human readable sentences that described what the results were.
The code was relatively short but the end effect was nice.
I've got a little app that you run and it dumps a GUID into the clipboard. You can run it /noui or not. With UI, its a single button that drops a new GUID every time you click it. Without it drops a new one and then exits.
I mostly use it from within VS. I have it as an external app and mapped to a shortcut. I'm writing an app that relies heavily on xaml and guids, so I always find I need to paste a new guid into xaml...
Any time I write a clever list comprehension or use of map/reduce in python. There was one like this:
if reduce(lambda x, c: locks[x] and c, locknames, True):
print "Sub-threads terminated!"
The reason I remember that is that I came up with it myself, then saw the exact same code on somebody else's website. Now-adays it'd probably be done like:
if all(map(lambda z: locks[z], locknames)):
print "ya trik"
I've got 20 or 30 of these things lying around because once I coded up the framework for my standard console app in windows I can pretty much drop in any logic I want, so I got a lot of these little things that solve specific problems.
I guess the ones I'm using a lot right now is a console app that takes stdin and colorizes the output based on xml profiles that match regular expressions to colors. I use it for watching my log files from builds. The other one is a command line launcher so I don't pollute my PATH env var and it would exceed the limit on some systems anyway, namely win2k.
I'm constantly connecting to various linux servers from my own desktop throughout my workday, so I created a few aliases that will launch an xterm on those machines and set the title, background color, and other tweaks:
alias x="xterm" # local
alias xd="ssh -Xf me#development_host xterm -bg aliceblue -ls -sb -bc -geometry 100x30 -title Development"
alias xp="ssh -Xf me#production_host xterm -bg thistle1 ..."
I have a bunch of servers I frequently connect to, as well, but they're all on my local network. This Ruby script prints out the command to create aliases for any machine with ssh open:
#!/usr/bin/env ruby
require 'rubygems'
require 'dnssd'
handle = DNSSD.browse('_ssh._tcp') do |reply|
print "alias #{reply.name}='ssh #{reply.name}.#{reply.domain}';"
end
sleep 1
handle.stop
Use it like this in your .bash_profile:
eval `ruby ~/.alias_shares`