I'm using Google Bigquery ML for the first time and try to train a linear regression model using the following command:
%%bigquery
CREATE OR REPLACE MODEL `sandbox.sample_lr_model`
OPTIONS
(model_type='linear_reg',
data_split_method ='no_split',
max_iterations=1) AS
SELECT
y AS label,
x AS x
FROM
`sandbox.y2018m08d01_rh_sample_dataframe_to_bq_v01_v01`
this step fails with the following error message:
ValueError: Table has no schema: call 'client.get_table()'
However the model is created and can be viewed:
The model has a so-called "Model schema". Am I doing something wrong?
google-cloud-bigquery==1.4.0 Python 3.5 Ubuntu
My input table is a minimum example:
This issue was fixed in https://github.com/GoogleCloudPlatform/google-cloud-python/pull/5602, which was released in version 1.4.0 of BigQuery. To double-check your BigQuery version, run !pip freeze | grep bigquery in a notebook cell.
Note that Datalab does not include the latest version of the google-cloud-bigquery library. To upgrade the version, run !pip install --upgrade google-cloud-bigquery.
Related
i git yolov7(https://github.com/WongKinYiu) with yolov7.pt and try to run
detect.py(i just want to run the example). it seems to be normal. but the output image has no mask.Why?
here is my code and log:
(PyTorch) E:\yolov7>python detect.py --weights yolov7.pt --source inference\images\bus.jpg
Namespace(weights=['yolov7.pt'], source='inference\\images\\bus.jpg', img_size=640, conf_thres=0.25, iou_thres=0.45, device='', view_img=False, save_txt=False, save_conf=False, nosave=False, classes=None, agnostic_nms=False, augment=False, update=False, project='runs/detect', name='exp', exist_ok=False, no_trace=False)
YOLOR v0.1-103-g6ded32c torch 1.11.0 CUDA:0 (NVIDIA GeForce GTX 1650, 4095.6875MB)
Fusing layers...
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
Model Summary: 306 layers, 36905341 parameters, 6652669 gradients
Convert model to Traced-model...
traced_script_module saved!
model is traced!
E:\anaconda\envs\PyTorch\lib\site-packages\torch\functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\TensorShape.cpp:2228.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
Done. (151.6ms) Inference, (9.3ms) NMS
The image with the result is saved in: runs\detect\exp4\bus.jpg
Done. (3.713s)
and here is my result:output image
You set the argument classes=None.
The classes variable refers to a list of classes, where you define the index of the entities saved inside the weights you are referencing for the inference.
From detect.py:
parser.add_argument('--classes', nargs='+', type=int, help='filter by class: --class 0, or --class 0 2 3')
Since you told the model to check for zero classes, the model itself will not report anything.
I was also facing this issue. After downgrading cuda version to 10.2 my problem was solved. I used Cuda 10.2 with PyTorch 1.10.0 via pip installation. I hope it helps you too.
pip3 install torch==1.10.0+cu102 torchvision==0.11.1+cu102 torchaudio===0.10.0+cu102 -f https://download.pytorch.org/whl/cu102/torch_stable.html
Source of the answer: link
Since when you are working with GPU, It allows half precision by default, which you can change by editing your detect.py file.
Go to detect.py file and not exactly sure but on line 31, you will see this line of code:
half = device.type != 'cpu' # half precision only supported on CUDA
Replace that line with
half = False
and then save default.py file.
Now when you are using your detection command, make sure to use --device 0 that indicates your GPU must be utilize for detection.
python detect.py --weights yolov7.pt --device 0 --source inference\images\bus.jpg
python mo_tf.py
--saved_model_dir C:\DATASETS\mask50000\exports\saved_model
--output_dir C:\DATASETS\mask50000
--reverse_input_channels
--tensorflow_custom_operations_config extensions\front\tf\mask_rcnn_support_api_v2.0.json
--tensorflow_object_detection_api_pipeline_config C:\DATASETS\mask50000\exports\pipeline.config
--log_level=DEBUG
I have been trying to convert the model using the above script, but every time I got the error:
"Exception: Exception occurred during running replacer "REPLACEMENT_ID (<class'extensions.front.tf.tensorflow_custom_operations_config_update.TensorflowCustomOperationsConfigUpdate'>)": The function 'update_custom_layer_attributes' must be implemented in the sub-class."
I have exported the graph using exporter_main_v2.py. If more information is needed please inform me.
EDIT:
I was able to convert the model by changing the file mask_rcnn_support_api_v2.4.json.
first change:
"custom_attributes": {
"operation_to_add": "Proposal",
"clip_before_nms": false,
"clip_after_nms": true
}
second change:
"start_points": [
"StatefulPartitionedCall/concat/concat",
"StatefulPartitionedCall/concat_1/concat",
"StatefulPartitionedCall/GridAnchorGenerator/Identity",
"StatefulPartitionedCall/Cast",
"StatefulPartitionedCall/Cast_1",
"StatefulPartitionedCall/Shape"
]
that solved the problme.
OpenVINO 2020.4 is not compatible with TensorFlow 2. Support for TF 2.0 Object Detection API models was fully enabled only in OpenVINO 2021.3.
I’ve successfully converted the model mask_rcnn_inception_resnet_v2_1024x1024_coco17 to IR using the latest OpenVINO release (2021.4.752).
I share the MO conversion command here:
python mo_tf.py --saved_model_dir <model_dir>\saved_model --tensorflow_object_detection_api_pipeline_config <pipeline_dir>\pipeline.config --transformations_config <installed_dir>\extensions\front\tf\mask_rcnn_support_api_v2.0.json"
We have finetuned our BERT model for text2text generation. It is working fine on the Jupyter notebook. But when I use the same trained model on another server of Ubuntu, then it shows the issue. This is my first post, so please bear with me. The issue I'm facing is that when I generate output on small sentences, it works fine. But on long sentences, it shows the following error:
At most 4 tokens in tensor([ 2, 2, 2, 2, 44763, 44763, 2, 44763]) can be equal to eos_token_id: 2. Make sure tensor([ 2, 2, 2, 2, 44763, 44763, 2, 44763]) are corrected.
My output generation code is:
from simpletransformers.seq2seq import Seq2SeqModel
#logging.basicConfig(level=logging.INFO)
#transformers_logger = logging.getLogger("transformers")
#transformers_logger.setLevel(logging.ERROR)
model = Seq2SeqModel(
encoder_decoder_type="bart", encoder_decoder_name="PATHOFMODEL",use_cuda=False,
)
while True:
original = input("Enter text to paraphrase: ")
to_predict = [original]
preds = model.predict(to_predict)
print("---------------------------------------------------------")
print(original)
print()
print("Predictions >>>")
for pred in preds[0]:
print(pred)
print("---------------------------------------------------------")
print()
You can check your TensorFlow version in your jupyter notebook and verify it on your Ubuntu server. If it's the same, there should be no error.
Firstly uninstall the tensorflow.
pip3 uninstall tensorflow
and then install any old version. Like in the case of version 2.2.0, then install 2.2.0 version using below command.
pip3 install tensorflow==2.2.0
It should work now.
I want to convert my 1 tensorflow model to IR currently I am following the instructions here:
https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html
The model I use is meta graph and ubuntu 16.04
I ran the line deflected:
python3 mo_tf.py --input_meta_graph .meta
then it will get an error:
[ERROR] Exception occurred during running replacer "None" (): Data flow edge coming out of AssignSub node model_0 / resnet_v1_50 / block4 / unit_1 / bottleneck_v1 / shortcut / BatchNorm / AssignMovingAvg
Can you guys please help me? thanks everyone
Did you freeze the model before conversion? Please look at how to freeze your model using instructions from https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py and retry.
I'm running the following basic code:
dfMain.to_csv('./January_filtered_International_WE.csv')
which used to run normally until yesterday. This morning I upgraded to pandas 0.25.0 while running code and now I cannot write my 500k rows dataframe to a csv. I can mention that I left Jupyter Notebook running in order to do some processing, so this morning when I opened it I had the dataFrame already, processed.
Versions (using Windows 10)
Jupyter notebook : 5.7.8
Python : 3.6.7
Pandas : 0.25.0
I would like to save my DataFrame in a fast and efficient manner as I will load it several times in the future. I do not want to close the notebook as this will delete the dataFrame.
I tried:
downgrading to Pandas 0.24.2 (previous version used) but still getting the __init__() got an unexpected keyword argument 'tupleize_cols'
use pd.to_pickle but got a memoryError
use pd.to_hdf but got a memoryError
using msgbox instead but apparently it does not support DataFrames (got an error)
upgrade Jupyter notebook, but got the following error:
ERROR: ipython 5.8.0 has requirement prompt-toolkit<2.0.0,>=1.0.4, but
you'll have prompt-toolkit 2.0.9 which is incompatible
so naturally I did pip install prompt-toolkit 1.0.16 but then got this message:
ERROR: jupyter-console 6.0.0 has requirement prompt-toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.16 which is incompatible.
As an alternative I went into PyCharm and took a random DataFrame.to_csv and it worked. This makes me think the issue is with Jupyter Notebook.
Any help on how to save the DataFrame (~12 GB) is appreciated!
Re-installing Jupyter did the trick in my case
I kept getting the same error, but updating Jupyter fixed it