I need to write a "standalone" script in Python to upload sales taxes to the account_tax table in the database using ONLY the ORM module of OpenERP. What I would like to do is something like the pseudo code below.
Can someone provide me a more details on the following:
1) what sys.path's do I need to set
2) what modules do I need to import before importing the "account" module. Currently when I import the "account" module I get the following error:
AssertionError: The report "report.custom" already exists!
3) What is the proper way to get my database cursor. In the code below I am simply calling psycopg2 directly to get a cursor.
If this approach cannot work, can anyone suggest an alternative approach other than writing XML files to load the data from the OpenERP application itself. This process needs to run outside of the the standard OpenERP application.
PSEUDO CODE:
import sys
# set Python paths to access openerp modules
sys.path.append("./openerp")
sys.path.append("./openerp/addons")
# import OpenERP
import openerp
# import the account addon modules that contains the tables
# to be populated.
import account
# define connection string
conn_string2 = "dbname='test2' user='xyz' password='password'"
# get a db connection
conn = psycopg2.connect(conn_string2)
# conn.cursor() will return a cursor object
cursor = conn.cursor()
# and finally use the ORM to insert data into table.
If you wanna do it via web service then have look at the OpenERP XML-RPC Web services
Example code top work with OpenERP Web Services :
import xmlrpclib
username = 'admin' #the user
pwd = 'admin' #the password of the user
dbname = 'test' #the database
# OpenERP Common login Service proxy object
sock_common = xmlrpclib.ServerProxy ('http://localhost:8069/xmlrpc/common')
uid = sock_common.login(dbname, username, pwd)
#replace localhost with the address of the server
# OpenERP Object manipulation service
sock = xmlrpclib.ServerProxy('http://localhost:8069/xmlrpc/object')
partner = {
'name': 'Fabien Pinckaers',
'lang': 'fr_FR',
}
#calling remote ORM create method to create a record
partner_id = sock.execute(dbname, uid, pwd, 'res.partner', 'create', partner)
More clearly you can also use the OpenERP Client lib
Example Code with client lib :
import openerplib
connection = openerplib.get_connection(hostname="localhost", database="test", \
login="admin", password="admin")
user_model = connection.get_model("res.users")
ids = user_model.search([("login", "=", "admin")])
user_info = user_model.read(ids[0], ["name"])
print user_info["name"]
You see both way are good but when you use the client lib, code is less and easy to understand while using xmlrpc proxy is lower level calls that you will handle
Hope this will help you.
As per my view one must go for XMLRPC or NETSVC services provided by Open ERP for such needs.
You don't need to import accounts module of Open ERP, there are possibilities that other modules have inherited accounts.tax object and had altered its behaviour as per your business needs.
Eventually if you feed data by calling those methods manually without using Open ERP Web service its possible you'll get undesired result / unexpected failures / inconsistent database state.
You can use Erppeek to browse data, but not sure if you can really upload data to DB, personally I use/prefer XMLRPC
Why don't you use the xmlrpc call of openerp.
it will not need to import account or openerp . and even you can have all orm functionality.
You can use python library to access openerp server using xmlrpc service.
Please check https://github.com/OpenERP/openerp-client-lib
It is officially supported by OpenERP SA.
If you want to interacti directly with the DB, you could just import psycopg2 and:
conn = psycopg2.connect(dbname='dbname', user='dbuser', password='dbpassword', host='dbhost')
cur = conn.cursor()
cur.execute('select * from table where id = %d' % table_id)
cur.execute('insert into table(column1, column2) values(%d, %d)' % (value1, value2))
cur.close()
conn.close()
Why you want to fix it like that?! You should create a localization module and define data in XML files. This is the standard way to fix such a problem in OpenERP.
You want to insert sales taxes for which country? Explain more plz.
from openerp.modules.registry import RegistryManager
registry = RegistryManager.get("databasename")
with registry.cursor() as cr:
user = registry.get('res.users').browse(cr, userid, listids)
print user
Related
I'm writing a small Flask application and am having it connect to Rserve using pyRserve. I want every session to initiate and then maintain its own Rserve connection.
Something like this:
session['my_connection'] = pyRserve.connect()
doesn't work because the connection object is not JSON serializable. On the other hand, something like this:
flask.g.my_connection = pyRserve.connect()
doesn't work because it does not persist between requests. To add to the difficulty, it doesn't seem as though pyRserve provides any identifier for a connection, so I can't store a connection ID in the session and use that to retrieve the right connection before each request.
Is there a way to accomplish having a unique connection per session?
The following applies to any global Python data that you don't want to recreate for each request, not just rserve, and not just data that is unique to each user.
We need some common location to create an rserve connection for each user. The simplest way to do this is to run a multiprocessing.Manager as a separate process.
import atexit
from multiprocessing import Lock
from multiprocessing.managers import BaseManager
import pyRserve
connections = {}
lock = Lock()
def get_connection(user_id):
with lock:
if user_id not in connections:
connections[user_id] = pyRserve.connect()
return connections[user_id]
#atexit.register
def close_connections():
for connection in connections.values():
connection.close()
manager = BaseManager(('', 37844), b'password')
manager.register('get_connection', get_connection)
server = manager.get_server()
server.serve_forever()
Run it before starting your application, so that the manager will be available:
python rserve_manager.py
We can access this manager from the app during requests using a simple function. This assumes you've got a value for "user_id" in the session (which is what Flask-Login would do, for example). This ends up making the rserve connection unique per user, not per session.
from multiprocessing.managers import BaseManager
from flask import g, session
def get_rserve():
if not hasattr(g, 'rserve'):
manager = BaseManager(('', 37844), b'password')
manager.register('get_connection')
manager.connect()
g.rserve = manager.get_connection(session['user_id'])
return g.rserve
Access it inside a view:
result = get_rserve().eval('3 + 5')
This should get you started, although there's plenty that can be improved, such as not hard-coding the address and password, and not throwing away the connections to the manager. This was written with Python 3, but should work with Python 2.
I have developed a script which executes against one DB instance e.g.: db1. The code to connect to DB is written in Background section. Now what i want to do is, i have to execute same test script against diffrent db instance e.g.:db2
Feature:Execution against multiple DB instance.
##############################################
Background:
* def db_properties = {db_username,db_password,db_connection_string,driver}
* def createConnection = path to read .java file
* def readFromDB = new createConnection(db_properties)
##############################################
In * def db_properties, i have hard coded the actual values of username, password, conenction string and driver.What exactly i want to do is, i have to validate my API response agains't another DB instance e.g. build is deployed in another environment, and db properties which i have mentioned is diffrent environment. How can i do it?
This has nothing to do with Karate. Maybe the solution is to have 2 sets of DB connection values in your karate-config.js. Please figure out a solution that is appropriate for your situation.
We are having one django rest framework (DRF) project which should have multiple databases (mongoDB).Each databases should be independed. We are able to connect to one database, but when we are going to another DB for writing connection is happening but data is storing in DB which is first connected.
We changed default DB and everything but no changes.
(Note : Solution should be apt for the usage of serializer. Because we need to use DynamicDocumentSerializer in DRF-mongoengine.
Thanks in advance.
While running connect() just assign an alias for each of your databases and then for each Document specify a db_alias parameter in meta that points to a specific database alias:
settings.py:
from mongoengine import connect
connect(
alias='user-db',
db='test',
username='user',
password='12345',
host='mongodb://admin:qwerty#localhost/production'
)
connect(
alias='book-db'
db='test',
username='user',
password='12345',
host='mongodb://admin:qwerty#localhost/production'
)
models.py:
from mongoengine import Document
class User(Document):
name = StringField()
meta = {'db_alias': 'user-db'}
class Book(Document):
name = StringField()
meta = {'db_alias': 'book-db'}
I guess, I finally get what you need.
What you could do is write a really simple middleware that maps your url schema to the database:
from mongoengine import *
class DBSwitchMiddleware:
"""
This middleware is supposed to switch the database depending on request URL.
"""
def __init__(self, get_response):
# list all the mongoengine Documents in your project
import models
self.documents = [item for in dir(models) if isinstance(item, Document)]
def __call__(self, request):
# depending on the URL, switch documents to appropriate database
if request.path.startswith('/main/project1'):
for document in self.documents:
document.cls._meta['db_alias'] = 'db1'
elif request.path.startswith('/main/project2'):
for document in self.documents:
document.cls._meta['db_alias'] = 'db2'
# delegate handling the rest of response to your views
response = get_response(request)
return response
Note that this solution might be prone to race conditions. We're modifying a Documents globally here, so if one request was started and then in the middle of its execution a second request is handled by the same python interpreter, it will overwrite document.cls._meta['db_alias'] setting and first request will start writing to the same database, which will break your database horribly.
Same python interpreter is used by 2 request handlers, if you're using multithreading. So with this solution you can't start your server with multiple threads, only with multiple processes.
To address the threading issues, you can use threading.local(). If you prefer context manager approach, there's also a contextvars module.
So I created a model for storing credentials from Gmail users.
I wanted to make migrations but it says that there is no such table:
django.db.utils.OperationalError: no such table: mainApp_credentialsmodel
My models:
from django.db import models
# Create your models here.
from django.contrib.auth.models import User
from django.db import models
import json
class CredentialsModel(models.Model):
id = models.ForeignKey(User, primary_key=True,on_delete=models.CASCADE)
credential = models.CharField(max_length=1000)
Calling that model for checking authorization:
SCOPES = 'https://www.googleapis.com/auth/gmail.readonly'
store = CredentialsModel.objects.all()
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('mainApp/client_secret.json', SCOPES)
creds = tools.run_flow(flow, store)
service = build('gmail', 'v1', http=creds.authorize(Http()))
python manage.py makemigrations
If that error keep happening, check your migrations folder and check the files inside. Also check If your database is online, in case you have a database online, I've got this problem last week, but it was a problem with azure.
In last case I would create the table (model) again, changing the name to something similar, but If you have a significant amount of data in that table, then I think you can't do that.
It looks like your authorization code - including the query on CredentialsModel - is at module level. This means it runs when the module is imported, which happens before the migration has had a chance to run.
You must ensure that any database-accessing code is inside a function or method and is not invoked globally.
Does anyone knows how to load initial data for auth.User using sql fixtures?
For my models, I just got have a < modelname >.sql file in a folder named sql that syncdb does it's job beautifully. But I have no clue how to do it for the auth.User model. I've googled it, but with no success.
Thanks in advance,
Aldo
For SQL fixtures, you'd have to specifically have insert statements for the auth tables. You can find the schema of the auth tables with the command python manage.py sql auth.
The much easier and database-independent way (unless you have some additional SQL magic you want to run), is to just make a JSON or YAML fixture file in the fixtures directory of your app with data like this:
- model: auth.user
pk: 100000
fields:
first_name: Admin
last_name: User
username: admin
password: "<a hashed password>"
You can generate a hashed password quickly in a django shell
>>> from django.contrib.auth.models import User
>>> u = User()
>>> u.set_password('newpass')
>>> u.password
'sha1$e2fd5$96edae9adc8870fd87a65c051e7fdace6226b5a8'
This will get loaded whenever you run syncdb.
You are looking for loaddata:
manage.py loadata path/to/your/fixtureFile
But I think the command can only deal with files in XML, YAML, Python or JSON format (see here). To create such appropriate files, have a look at the dumpdata method.
Thanks for your answers. I've found the solution that works for me, and for coincidence was one of Brian's suggestion. Here it is:
Firs I disconnected the signal that created the Super User after syncdb, for I have my super user in my auth_user fixture:
models.py:
from django.db.models import signals
from django.contrib.auth.management import create_superuser
from django.contrib.auth import models as auth_app
signals.post_syncdb.disconnect(
create_superuser,
sender=auth_app,
dispatch_uid = "django.contrib.auth.management.create_superuser")
Then I created a signal to be called after syncdb:
< myproject >/< myapp >/management/__init__.py
"""
Loads fixtures for files in sql/<modelname>.sql
"""
from django.db.models import get_models, signals
from django.conf import settings
import <myproject>.<myapp>.models as auth_app
def load_fixtures(app, **kwargs):
import MySQLdb
db=MySQLdb.connect(host=settings.DATABASE_HOST or "localhost", \
user=settings.DATABASE_USER,
passwd=settings.DATABASE_PASSWORD, port=int(settings.DATABASE_PORT or 3306))
cursor = db.cursor()
try:
print "Loading fixtures to %s from file %s." % (settings.DATABASE_NAME, \
settings.FIXTURES_FILE)
f = open(settings.FIXTURES_FILE, 'r')
cursor.execute("use %s;" % settings.DATABASE_NAME)
for line in f:
if line.startswith("INSERT"):
try:
cursor.execute(line)
except Exception, strerror:
print "Error on loading fixture:"
print "-- ", strerror
print "-- ", line
print "Fixtures loaded"
except AttributeError:
print "FIXTURES_FILE not found in settings. Please set the FIXTURES_FILE in \
your settings.py"
cursor.close()
db.commit()
db.close()
signals.post_syncdb.connect(load_fixtures, sender=auth_app, \
dispatch_uid = "<myproject>.<myapp>.management.load_fixtures")
And in my settings.py I added FIXTURES_FILE with the path to my .sql file with the sql dump.
One thing that I still haven't found is how to fire this signal only after the tables are created, and not everytime syncdb is fired. A temporary work around for this is use INSERT IGNORE INTO in my sql command.
I know this solution is far from perfect, and critics/improvements/opinions are very welcome!
Regards,
Aldo
There is a trick for this: (tested on Django 1.3.1)
Solution:
python manage.py startapp auth_fix
mkdir auth_fix/fixtures
python manage.py dumpdata auth > auth_fixtures/fixtures/initial_data.json
Include auth_fix in INSTALLED_APPS inside settings.py
Next time you run python manage.py syncdb, Django will load the auth fixture automatically.
Explanation:
Just make an empty app to hold the fixtures folder. Leave __init__py, models.py and views.py in it so that Django recognizes it as an app and not just a folder.
Make the fixtures folder in the app.
python manage.py dumpdata auth will dump the "auth" data in the DB with all the Groups and Users information. The rest of the command simply redirects the output into a file called "initial_data.json" which is the one that Django looks for when you run "syncdb".
Just include auth_fix in INSTALLED_APPS inside settings.py.
This example shows how to do it in JSON but you can basically use the format of your choice.
If you happen to be doing database migrations with south, creating users is very simple.
First, create a bare data migration. It needs to be included in some application. If you have a common app where you place shared code, that would be a good choice. If you have an app where you concentrate user-related code, that would be even better.
$ python manage.py datamigration <some app name> add_users
The pertinent migration code might look something like this:
# encoding: utf-8
import datetime
from south.db import db
from south.v2 import DataMigration
from django.db import models
from django.contrib.auth.models import User
class Migration(DataMigration):
users = [
{
'username': 'nancy',
'email': 'nancy#example.com',
'password': 'nancypassword',
'staff': True,
'superuser': True
},
{
'username': 'joe',
'email': '',
'password': 'joepassword',
'staff': True,
'superuser': False
},
{
'username': 'susan',
'email': 'susan#example.com',
'password': 'susanpassword',
'staff': False,
'superuser': False
}
]
def forwards(self, orm):
"""
Insert User objects
"""
for i in Migration.users:
u = User.objects.create_user(i['username'], i['email'], i['password'])
u.is_staff = i['staff']
u.is_superuser = i['superuser']
u.save()
def backwards(self, orm):
"""
Delete only these users
"""
for i in Migration.users:
User.objects.filter(username=i['username']).delete()
Then simply run the migration and the auth users should be inserted.
$ python manage.py migrate <some app name>
An option is to import your auth.User SQL manually and subsequently dump it out to a standard Django fixture (name it initial_data if you want syncdb to find it). You can generally put this file into any app's fixtures dir since the fixtured data will all be keyed with the proper app_label. Or you can create an empty/dummy app and place it there.
Another option is to override the syncdb command and apply the fixture in a manner as you see fit.
I concur with Felix that there is no non-trivial natural hook in Django for populating contrib apps with SQL.
I simply added SQL statements into the custom sql file for another model. I chose my Employee model because it depends on auth_user.
The custom SQL I wrote actually reads from my legacy application and pulls user info from it, and uses REPLACE rather than INSERT (I'm using MySQL) so I can run it whenever I want.
And I put that REPLACE...SELECT statement in a procedure so that it's easy to run manually or scheduled with cron.