i want to backup /var/lib/libvirt/backups/onlyoffice folder with borgbackup, it has some xml files and some snapshots (.bimg), as well as qcow2 image.
the image weighs 14 GB.
root#kvm610:~ # ls -lh /var/lib/libvirt/backups/onlyoffice/onlyoffice.qcow2
-rwxr-xr-x 1 libvirt-qemu kvm 14G avril 12 07:17 /var/lib/libvirt/backups/onlyoffice/onlyoffice.qcow2*
the first borgbackup backup lasts 2m37, all the files are tagged A, because "added".
root#kvm610:~ # time borg create --list /var/lib/libvirt/borgbackup/kvm610::trial4 /var/lib/libvirt/backups/onlyoffice
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-28_06:25:22.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-23_06:25:35.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-09-02_06:25:03.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-26_06:25:35.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-29_06:25:14.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-30_06:25:17.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-31_06:25:26.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-09-01_06:25:10.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-22_06:25:08.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-24_06:25:51.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-25_06:25:23.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-27_06:25:14.xml
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-141702
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-131701
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-121701
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-111701
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-101701
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-091701
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-081701
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-071702
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.qcow2
d /var/lib/libvirt/backups/onlyoffice
real 2m37,093s
user 1m46,503s
sys 0m11,726s
I then restart the same backup, in the same warehouse but with a different archive name (trial5). all the xml and bimg files are tagged U, like "unchanged", but the qcow2 image is still tagged A, even though it hasn't moved ... the VM has been shut down since April.
root#kvm610:~ # time borg create --list /var/lib/libvirt/borgbackup/kvm610::trial5 /var/lib/libvirt/backups/onlyoffice
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-28_06:25:22.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-23_06:25:35.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-09-02_06:25:03.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-26_06:25:35.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-29_06:25:14.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-30_06:25:17.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-31_06:25:26.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-09-01_06:25:10.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-22_06:25:08.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-24_06:25:51.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-25_06:25:23.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-27_06:25:14.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-141702
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-131701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-121701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-111701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-101701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-091701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-081701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-071702
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.qcow2
d /var/lib/libvirt/backups/onlyoffice
real 2m36,473s
user 1m45,359s
sys 0m11,831s
you have an idea to explain this?
and the backup time is the same ... 2m37 / 2m36
thanks in advance,
According to Thorsten Schöning, who answered my question on the list borgbackup#python.org, the answer is contained in the borgbackup documentation.
see I am seeing ‘A’ (added) status for an unchanged file
The files cache is used to determine whether Borg already “knows” / has backed up a file and if so, to skip the file from chunking. It intentionally excludes files that have a timestamp which is the same as the newest timestamp in the created archive.
...
If you want to avoid unnecessary chunking, just create or touch a small or empty file in your backup source file set (so that one has the latest timestamp, not your 50GB VM disk image) and, if you do snapshots, do the snapshot after that.
So i decided to add a new a file in the var/lib/libvirt/backups/onlyoffice folder: passwd. The first backup (with passwd) takes the same time as the previous one: 2m35
root#kvm610:/var/lib/libvirt/backups/onlyoffice # time borg create --list /var/lib/libvirt/borgbackup/kvm610::essai6 /var/lib/libvirt/backups/onlyoffice
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-28_06:25:22.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-23_06:25:35.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-09-02_06:25:03.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-26_06:25:35.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-29_06:25:14.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-30_06:25:17.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-31_06:25:26.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-09-01_06:25:10.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-22_06:25:08.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-24_06:25:51.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-25_06:25:23.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-27_06:25:14.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-141702
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-131701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-121701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-111701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-101701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-091701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-081701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-071702
A /var/lib/libvirt/backups/onlyoffice/onlyoffice.qcow2
A /var/lib/libvirt/backups/onlyoffice/passwd
d /var/lib/libvirt/backups/onlyoffice
real 2m34,957s
user 1m44,797s
sys 0m12,468s
But the next backup is immediate: 0m07 !!
root#kvm610:/var/lib/libvirt/backups/onlyoffice # time borg create --list /var/lib/libvirt/borgbackup/kvm610::essai7 /var/lib/libvirt/backups/onlyoffice
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-28_06:25:22.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-23_06:25:35.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-09-02_06:25:03.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-26_06:25:35.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-29_06:25:14.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-30_06:25:17.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-31_06:25:26.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-09-01_06:25:10.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-22_06:25:08.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-24_06:25:51.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-25_06:25:23.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice_2021-08-27_06:25:14.xml
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-141702
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-131701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-121701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-111701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-101701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-091701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-081701
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.bimg-20210412-071702
U /var/lib/libvirt/backups/onlyoffice/onlyoffice.qcow2
A /var/lib/libvirt/backups/onlyoffice/passwd
d /var/lib/libvirt/backups/onlyoffice
real 0m0,682s
user 0m0,454s
sys 0m0,146s
and the qcow2 file is tagged "U", as expected!
A big thank you to Thorsten Schöning.
Ernest Chiarello.
Related
enter image description here
I have two tables :
UserInfo
Skill
and the join table between them called UserSkill as you can see at the
right part of the diagram.
I want to know whoever knows or is skillful in Java, what else he is skillful at. I mean for example I know java, Go, PHP, python and user number 2 knows java and python and CSS. So the answer to the question: whoever knows java what else he knows would be GO, PHP, Python and CSS.
It's like recommendation systems for example whoever but this product what else do they bought? Like what we have in amazon ..
What would be the best query for this ?
Thank you
More information:
UserInfo
U-id U-name
1 A
2 B
3 C
SkillInfo
S-id S-Name
1 Java
2 GO
3 PHP
4 Python
5 CSS
UserSkill:
U-id S-id
1 1
1 2
1 3
1 4
2 1
2 4
2 5
In SQL Server 2017 and Azure SQL DB you can use the new graph database capabilities and the new MATCH clause to answer queries like this, eg
SELECT FORMATMESSAGE ( 'User %s has skill %s and other skill %s.', [user1].[U-name], mainSkill.[S-name], otherSkill.[S-name] ) result
FROM
dbo.users user1,
dbo.hasSkill hasSkill1, dbo.skills mainSkill, dbo.hasSkill hasSkill2, dbo.skills otherSkill
WHERE mainSkill.[S-name] = 'CSS'
AND otherSkill.[S-name] != mainSkill.[S-name]
AND MATCH ( mainSkill<-(hasSkill1)-user1-(hasSkill2)->otherSkill);
My results:
Obviously you can answer the same queries with a relational approach, it's just a different way of doing things. Full script available here.
To make this more dynamic, replace the hard coded 'java' with a variable that you can pass to filter by any skill type, possibly make a stored procedure so you can pass the variable,
Edited column names as I didn't look at the image you provided:
--Outer query selects all skills of users which are not java and user has skill of java,
--the inner query selects all user ids where the user has a skill of java
SELECT sk.[SkillName], ui.[UserName], ui.[UserId]
FROM [dbo].[Skill] AS sk
INNER JOIN [dbo].[UserSkill] AS us
ON us.[SkillId] = sk.[SkillId]
INNER JOIN [dbo].[UserInfo] AS ui
ON ui.[UserId] = us.[UserId]
WHERE sk.[Skill] <> 'java' AND ui.[UserId] IN (
SELECT [UserId]
FROM [dbo].[UserInfo] ui
INNER JOIN [dbo].[UserSkill] us
ON us.[UserId] = ui.[UserId]
INNER JOIN [dbo].[Skill] sk
ON sk.[SkillId] = us.[SkillId]
WHERE sk.[SkillName] = 'java')
This is what I have found
--Formatted query
select
o.UserName, k.SkillName
from
UserSkill S
inner join
UserSkill SS on s.UserID = ss.UserID
and s.SkillID = 1
and ss.SkillID <> 1
inner join
Skill k on k.SkillID = ss.SkillID
inner join
UsersINFO O on o.UserID = ss.UserID
I have a strange request from the business. They want a report where they give a tcode or tcodes and the program will check which department has this/these and which users.
OK, I have found a couple of tables like AGR_TCODES (Assignment of roles to Tcodes), AGR_USERS (Assignment of roles to users) and USER_ADDR (Users by address data) to find what I want.
My question is: if a user has access to a tcode that it does not belong to one of his role, how can we catch this?
For example: I have access to VA03 but none of my roles is connected to this tcode.
Is there any way to catch this?
Try transaction S_BCE_68001397 (user selection based on permission values), select for permission object S_TCODE and the transaction as the permission value should give you the list you need. Transaction S_BCE_68001398 does the same, but doesn't allow you to query users for other permission objects.
The problem with these reports is that it might be either difficult or tedious to check for a large number of transaction codes. You could have a look at the source code of those transactions, report RSUSR002 and see if you can use the locally defined classes the reports use to query for multiple transaction codes and create a report based on your specific requirements.
Yoy can use tables AGR_1251 and AGR_USERS.
AGR_1251 will give you the Roles with the S_TCODE object and the Value = tcode given by the user.
Then you go to table AGR_USERS and get the users for the role with the S_TCODE= tcode.
SELECT adr_users~UNAME
FROM AGR_USERS JOIN AGR_1251 ON AGR_USERS~MANDT = AGR_1251~MANDT
and AGR_USERS~AGR_NAME = AGR_1251~AGR_NAME
WHERE AGR_1251~OBJECT = 'S_TCODE'
AND AGR_1251~LOW = param_tcode.
Hope this helps
Finally with the help of Dirk Trilsbeek I found the solution to what I was looking for. Here is the selection:
SELECT DISTINCT a~von e~ttext d~department d~bname d~name_first
d~name_last d~name_textc c~profile
INTO CORRESPONDING FIELDS OF TABLE gt_tcode_per_dprtm_usr
FROM ust12 AS a
INNER JOIN ust10s AS b
ON a~auth = b~auth AND
a~objct = b~objct AND
a~aktps = b~aktps
INNER JOIN ust04 AS c
ON b~profn = c~profile
INNER JOIN user_addr AS d
ON c~bname = d~bname
INNER JOIN tstct AS e
ON e~tcode = a~von
WHERE a~objct = 'S_TCODE' AND
a~von IN so_tcode AND
e~sprsl = 'G'.
I want to thank all of you for your answers.
I have two tables with attachments stored in my databse. People from multiple buildings use my application to upload attachments to the database (the files themselves are stored in type bytea). I need to figure out the total size (in GB) of the attachments for each building.
SELECT sum(octet_length(attachmentfile))
FROM schema.tattachments a, schema.trequests r, schema.tlocations l
WHERE r.location = l.locationid
AND a.reqid = r.reqid
AND r.sys_actntm > '2014-09-18'
GROUP BY l.building
The above SQL returns a bigint result separated by building:
sum bigint
--|-----------
1 | 15782159611407981
2 | 1140653769
3 | 710849157667
etc...
How can I format this SQL statement so it will give me the info in GB or MB, rather than these large numbers?
PostgreSQL has some system administration functions that will take care of this for you. Something along these lines should work.
SELECT pg_size_pretty(sum(octet_length(attachmentfile)::bigint))
FROM schema.tattachments a, schema.trequests r, schema.tlocations l
WHERE r.location = l.locationid
AND a.reqid = r.reqid
AND r.sys_actntm > '2014-09-18'
GROUP BY l.building;
You might be better off replacing octet_length() with pg_column_size(). Not sure how that would affect your query.
Background: First post so go easy on me! ;-) I'm having some problems with a SQLite join across 5 tables. I'm building a watering system on my Raspberry Pi, using Python, with the following table structure built in SQLite:
deviceTable - table lists all watering devices
programTable - table listing all watering programs
zoneTable - table listing all watering zones
assignProgramTable - bridging table for assigning many watering program to many zones
assignDeviceTable - bridging table for assigning many devices to many watering zones.
Problem: I'm currently using the following query to try and join all of these tables together so that I can begin to control my Raspberry Pi with a multi-threaded Python script... I originally had this working as a nested SQL script (working dynamically in my Python script), but it got a little complex when trying to run multiple thread within a database results. Essentially I plan to take a number of key values from the SQL query and
There appears to be multiple rows being returned when there should only be one for each entry.. any idea of the best way to handle this? I've found a number of answers suggesting to use the DISTINCT option (which didn't work), although am I going about this the best way possible?
SELECT *
FROM deviceTable AS d, programTable AS p
JOIN assignDeviceTable AS ad ON d.deviceID = ad.deviceID
JOIN zoneTable AS z ON z.zoneID = a.zoneID
JOIN assignProgramTable AS a ON p.programID = a.programID
where p.programStatus = "1"
Following LS_dev's advice, I've updated the query to the following:
SELECT *
FROM deviceTable AS d, programTable AS p, assignDeviceTable AS ad, zoneTable AS z, assignProgramTable AS ap
JOIN deviceTable ON d.deviceID = ad.deviceID
JOIN assignDeviceTable ON ad.zoneID = z.zoneID
JOIN zoneTable ON z.zoneID = ap.zoneID
JOIN assignProgramTable ON ap.programID = p.programID
where p.programStatus = "1"
Unfortunately this resulted in even more rows being duplicated... still stuck!
In SQLite , is same as JOIN.
So, your query is same as shown below:
SELECT *
FROM deviceTable AS d
JOIN programTable AS p -- missing ON p.some_id=d.some_id
JOIN assignDeviceTable AS ad ON d.deviceID = ad.deviceID
JOIN zoneTable AS z ON z.zoneID = a.zoneID
JOIN assignProgramTable AS a ON p.programID = a.programID
where p.programStatus = "1"
You have 5 tables but just 3 join constraints. Normally Nconstraints = Ntables - 1.
Analysing your relations:
programTable p deviceTable d
/ \ / \
(p.programID=a.programID) (d.deviceID=ad.deviceID)
\ / \ /
assignProgramTable a assignDeviceTable ad
/ \
(z.zoneID=a.zoneID)
\ /
zoneTable z
is visible your missing a relation between those two groups. From your explanation I think it should be something like ad.zoneID = z.zoneID.
I am having difficulty with writing an SQL query for my application. Below shows a screenshot of some of the tables in my database.
Please let me explain how a section of my application works, and then I can show you the problem I am having with my SQL query.
A User can create an application form on my system (dbo.Form). An application form consists of several sections, one of those sections is Employment History (dbo.FormEmployment). An employment record includes details like employer name, start date etc, but also a gradeID. A User can add one or more employment records to the table dbo.FormEmployment.
A system administrator can add a Shift (dbo.Shift) to the system, and also then assign Grades to a Shift. These Grades are recorded in the dbo.ShiftGrade table. A Shift can be assigned 1 or more Grades, i.e. it could be assigned 1,2,3,4 or 5 Grades.
When the system administrator has added the Shift and Shift Grades, I then want to perform an SQL query to return a list of Users whose Grades match that of the Grades assigned to a Shift (remember the User adds their Grade when they add an employment record).
I have written the following SQL query which works, however, the issue with this query is that it returns a list of Users, that have any Grade that matches that of the Grades assigned to a Shift.
SELECT u.userID, u.userTypeID, u.firstName, u.lastName, u.email, u.password,
u.contactNumber, u.organisationID, u.emailVerificationCode, u.mobileVerificationCode,
u.userStatusID, u.AddedBy, u.AddedDate, u.ModifiedBy, u.ModifiedDate
FROM [User] u
--Check Grades Match
WHERE u.userID IN
(
SELECT f.locumID
FROM dbo.Form f, dbo.FormEmployment emp
WHERE f.formID = emp.formID
AND emp.statusID = 101
AND emp.workYN = 1
AND emp.gradeID IN (
select gradeID from dbo.ShiftGrade where shiftID = #p0
)
)
You can see I am using the IN statement to do this. However, I only want to return a list of Users who can match exactly the same Grades that have been assigned to a Shift. Therefore, if a Shift has been assigned 3 Grades, then I want a List of Users who also have the exact same three Grades to be returned.
I apologise for the lengthy post, I just thought it would be better explained this way.
Any feedback or help with this would be greatly appreciated.
select u.*
from dbo.Users u
join dbo.Form f on u.? = f.formId
join dbo.FormEmployment fe on fe.formId = f.formId
join dbo.Grade g on g.gradeId = fe.gradeId
join dbo.ShiftGrade shg on shg.gradeId =g.gradeId
join dbo.Shift sh on sh.shiftId = shg.shiftId
where
sh.shiftId = -- recently added shift id
and g.gradeId == -- recently added grade id