Raw SQL queries in TYPO3 9 - typo3-9.x

Is there a way to perform raw SQL queries in TYPO3 9?
Something equivalent to $GLOBALS['TYPO3_DB']->sql_query($sql); in previous versions.

In your repository, you can use statement() for this.
Example:
$query = $this->createQuery();
$sql = '
SELECT fieldA, fieldB
FROM table
WHERE
pid = '.$pid.'
AND someField = 'something'
';
$query->statement($sql)->execute();
Make sure, that you take care of sanitizing the input!

THE ORM-concept seems to makes it difficult using raw SQL.
/**
* #param string $tableName
* #return bool
*/
public function createMySplitTable($newTableName = self::TABLENAME)
{
if ($newTableName !== self::TABLENAME) {
$baseTable = self::TABLENAME;
// make a structure-copy of the main table
$sql ="CREATE TABLE $newTableName SELECT * FROM $baseTable AS main LIMIT 0;";
// looky-looky at 20200609: https://www.strangebuzz.com/en/snippets/running-raw-sql-queries-with-doctrine
// seems to work
/** #var Connection $connection */
$connection = GeneralUtility::makeInstance(ConnectionPool::class)
->getConnectionForTable(self::TABLENAME);
/** #var DriverStatement $statement */
$statement = $connection->prepare($sql);
$statement->execute();
// // --- don't work for me :-(
// /** #var QueryBuilder $queryBuilder */
// $queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
// ->getQueryBuilderForTable($baseTable);
// $queryBuilder->resetRestrictions();
// $queryBuilder->resetQueryParts();
// $queryBuilder->add($sql,'select'); // Tried some variations
// $queryBuilder->execute();
// // --- don't work for me :-(
// /** #var Query $query */ // Extbase won't work for this query
// $query = $this->createQuery();
// $query->statement($sql);
// $query->execute(true);
// // --- work only for TYPO3 8 and lower
// $GLOBALS['TYPO3_DB']->sql_query($sql); /// < TYPO3 8
//
}
}
Thanks to https://www.strangebuzz.com/en/snippets/running-raw-sql-queries-with-doctrine

you could use the querybuilder with it's method
TYPO3\CMS\Core\Database\Query\QueryBuilder::selectLiteral(string ... $selects).
Be aware:
Specifies items that are to be returned in the query result. Replaces any previously specified selections, if any. This should only be used for literal SQL expressions as no quoting/escaping of any kind will be performed on the items.
There also is
TYPO3\CMS\Core\Database\Query\QueryBuilder::addSelectLiteral(string ... $selects)
Adds an item that is to be returned in the query result. This should only be used for literal SQL expressions as no quoting/escaping of any kind will be performed on the items.

Related

ActiveJDBC , How can i query some columns i interest with in a single table

when i query a single table , i do not want all columns , i just want some column that i interest in.
For example, when i use where method to query a table, it will query all columns in a table like
public class SubjectSpecimenType extends Model {
}
SubjectSpecimenType.where("SUBJECT_ID = ? AND SITE_ID = ?", subjectId, siteId);
i don't know if there has a method named select that i can use to query some column like
SubjectSpecimenType.select("SUBJECT_NAME", "SITE_NAME").where("SUBJECT_ID = ? AND SITE_ID = ?", subjectId, siteId);
there are the source code in LazyList.java
/**
* Use to see what SQL will be sent to the database.
*
* #param showParameters true to see parameter values, false not to.
* #return SQL in a dialect for current connection which will be used if you start querying this
* list.
*/
public String toSql(boolean showParameters) {
String sql;
if(forPaginator){
sql = metaModel.getDialect().formSelect(null, null, fullQuery, orderBys, limit, offset);
}else{
sql = fullQuery != null ? fullQuery
: metaModel.getDialect().formSelect(metaModel.getTableName(), null, subQuery, orderBys, limit, offset);
}
if (showParameters) {
StringBuilder sb = new StringBuilder(sql).append(", with parameters: ");
join(sb, params, ", ");
sql = sb.toString();
}
return sql;
}
when call formSelect method, the second param columns always be null
is there a unfinish TODO ?
When operating on Models, ActiveJDBC always selects all columns, because if you load a model and it has partial attributes loaded, then you have a deficient model. The columns are specified in some edge cases, as in the RawPaginator: https://github.com/javalite/javalite/blob/e91ebdd1e4958bc0965d7ee99e6b7debc59a7b85/activejdbc/src/main/java/org/javalite/activejdbc/RawPaginator.java#L141
There is nothing to finish here, the behavior is intentional.

Symfony 4 serialize entity without relations

I have to log the changes of each entity. I've Listener which listens for doctrine's events on preRemove, postUpdate and postDelete.
My entity AccessModule has relations:
App\Entity\AccessModule.php
/**
* #ORM\OneToMany(targetEntity="App\Entity\AccessModule", mappedBy="parent")
* #ORM\OrderBy({"id" = "ASC"})
*/
private $children;
/**
* #ORM\ManyToOne(targetEntity="App\Entity\AccessModule", inversedBy="children")
* #ORM\JoinColumn(name="parent_id", referencedColumnName="id", nullable=true)
*/
private $parent;
/**
* #ORM\ManyToMany(targetEntity="App\Entity\AccessModuleRoute", inversedBy="access_modules")
* #ORM\JoinTable(name="access_routes",
* joinColumns={#ORM\JoinColumn(name="access_module_id", referencedColumnName="id")},
* inverseJoinColumns={#ORM\JoinColumn(name="route_id", referencedColumnName="id")})
*
*/
private $routes;
in listener:
App\EventListener\EntityListener.php
use Symfony\Component\Serializer\Encoder\JsonEncoder;
use Symfony\Component\Serializer\Encoder\XmlEncoder;
use Symfony\Component\Serializer\Normalizer\ObjectNormalizer;
$encoders = [new XmlEncoder(), new JsonEncoder()];
$normalizer = new ObjectNormalizer();
$normalizer->setCircularReferenceHandler(function ($object) {
return $object->getId();
});
$this->serializer = new Serializer([$normalizer], $encoders);
public function createLog(LifecycleEventArgs $args, $action){
$em = $args->getEntityManager();
$entity = $args->getEntity();
if ($this->tokenStorage->getToken()->getUser()) {
$username = $this->tokenStorage->getToken()->getUser()->getUsername();
} else {
$username = 'anon'; // TODO Remove anon. set null value
}
$log = new Log();
// $log->setData('dddd')
$log->setData($this->serializer->serialize($entity, ''json)
->setAction($action)
->setActionTime(new \DateTime())
->setUser($username)
->setEntityClass(get_class($entity));
$em->persist($log);
$em->flush();
}
I've problem with serialization
When I use $log->setData($entity) I get problem with Circular.
Whan I do serialization $log->setData($this->serializer->serialize($entity, ''json) I get full of relations, with parent's children, with children children. In a result I get full tree :/
I'd like to get
Expect
[
'id' => ID,
'name' => NAME,
'parent' => parent_id // ManyToOne, I'd like get its id
'children' => [$child_id, $child_id, $child_id] // array of $id of children array collection
]
(ofcourse this is draft before encode it to json)
How can I get expected data without full relations?
Thing you are looking for is called serialization groups: here and here.
Now let me explain how it works. It's quite simple. Say you have Post Entity:
class Post
{
/**
* #ORM\Id
* #ORM\GeneratedValue
* #ORM\Column(type="integer")
* #Groups({"default"})
*/
private $id;
/**
* #ORM\ManyToOne(targetEntity="App\Entity\User\User")
* #Groups({"default"})
*/
private $author;
}
And you have also User Entity:
class User
{
/**
* #ORM\Id
* #ORM\GeneratedValue
* #ORM\Column(type="integer")
* #Groups({"default"})
*/
private $id;
/**
* #ORM\Column(type="string", length=40)
* #Groups({"default"})
*/
private $firstName;
/**
* #ORM\Column(type="string", length=40)
*/
private $lastName;
}
Post can have author(User) but I don't want to return all User data every single time. I'm interested only in id and first name.
Take a closer look at #Groups annotation. You can specify so called serialization groups. It's nothing more than convinient way of telling Symfony which data you would like to have in your result set.
You have to tell Symfony serializer which relationships you would like to keep by adding relevant groups in form of annotation above property/getter. You also have to specify which properties or getters of your relationships you would like to keep.
Now how to let Symfony know about that stuff?
When you prepare/configure your serializaition service you just simply have to provide defined groups like that:
return $this->serializer->serialize($data, 'json', ['groups' => ['default']]);
It's good to build some kind of wrapper service around native symfony serializer so you can simplify the whole process and make it more reusable.
Also make sure that serializer is correctly configured - otherwise it will not take these group into account.
That is also one way(among other ways) of "handling" circular references.
Now you just need to work on how you will format your result set.
ignored_attributes provides a quick, and easy way to accomplish what you're looking for.
$serializer->serialize($object, 'json', ['ignored_attributes' => ['ignored_property']]);
Heres how its used with the serializer:
use Acme\Person;
use Symfony\Component\Serializer\Encoder\JsonEncoder;
use Symfony\Component\Serializer\Normalizer\ObjectNormalizer;
use Symfony\Component\Serializer\Serializer;
$person = new Person();
$person->setName('foo');
$person->setAge(99);
$normalizer = new ObjectNormalizer();
$encoder = new JsonEncoder();
$serializer = new Serializer([$normalizer], [$encoder]);
$serializer->serialize($person, 'json', ['ignored_attributes' => ['age']]);
Documentation: https://symfony.com/doc/current/components/serializer.html#ignoring-attributes
Tested in Symfony 4.1, here is the documentation that actually works https://symfony.com/blog/new-in-symfony-2-7-serialization-groups
Robert's explanation https://stackoverflow.com/a/48756847/579646 is missing the $classMetadataFactory in order to work. Here is my code:
$classMetadataFactory = new ClassMetadataFactory(new AnnotationLoader(new AnnotationReader()));
$encoders = [new JsonEncoder()];
$normalizer = new ObjectNormalizer($classMetadataFactory);
$normalizer->setCircularReferenceLimit(2);
// Add Circular reference handler
$normalizer->setCircularReferenceHandler(function ($object) {
return $object->getId();
});
$normalizers = [$normalizer];
$serializer = new Serializer($normalizers, $encoders);
$jsonContent = $serializer->serialize($jobs, 'json', array('groups' => ['default']));
return JsonResponse::fromJsonString($jsonContent);

Extending news Model does not work for FE and news_ttnewsimport

In order to upgrade an existing system, I have to import extended tt_news records to tx_news. The problem is, that the extending of the tx_news Model seems not to work proper and of course this, the import neither.
But in Backend I can see and store data in my additional fields.
What I've done so far:
I've extended tx_news Version 3.2.8
My Model:
class News extends \GeorgRinger\News\Domain\Model\News {
/**
* uidForeign.
*
* #var int
*/
protected $uidForeign;
/**
* Sets the uidForeign.
*
* #param int $uidForeign
*
* #return void
*/
public function setUidForeign($uidForeign)
{
$this->uidForeign = $uidForeign;
}
/**
* Returns the uidForeign.
*
* #return int $uidForeign
*/
public function getUidForeign()
{
return $this->uidForeign;
}
/**
* tableForeign.
*
* #var string
*/
protected $tableForeign;
/**
* Sets the tableForeign.
*
* #param string $tableForeign
*
* #return void
*/
public function setTableForeign($tableForeign)
{
$this->tableForeign = $tableForeign;
}
/**
* Returns the tableForeign.
*
* #return string $tableForeign
*/
public function getTableForeign()
{
return $this->tableForeign;
}
}
ext_localconf:
$GLOBALS['TYPO3_CONF_VARS']['EXT']['news']['classes']['Domain/Model/News'][] = 'news_extend';
I think that should work. The generated class in typo3temp seems correct. My fields including their getter/setter are in there.
But in Controller and FE I can not access these fields.
What am I missing? What else can I check?
If you upgrade a project, I don't know really the reason why you are using an old version of EXT:news.
What could be missing is the TCA definition of the field.
If you want to migrate from tt_news to news, there is a ready-to-use solution which can be found here https://github.com/ext-news/news_ttnewsimport
The reason was an configuration setting for the backend cache.
they have bees set to TYPO3\CMS\Core\Cache\Backend\NullBackend:class instead TYPO3\CMS\Core\Cache\Backend\NullBackend.
Now it works.

In TYPO3 Extbase Repository use statement and like together

Following function works. But I wanna use "statement" and "like" together. If I comment out the line with constraint1 and try with:
$constraint1 = $query->statement("SELECT * FROM pages WHERE title LIKE '" . $text . "%'");
...the query doesn't work.
I got an error:
1: PHP Catchable Fatal Error: Argument 1 passed to TYPO3\CMS\Extbase\Persistence\Generic\Qom\QueryObjectModelFactory::_or() must implement interface TYPO3\CMS\Extbase\Persistence\Generic\Qom\ConstraintInterface, instance of TYPO3\CMS\Extbase\Persistence\Generic\Query given, called in [...] on line 439 and defined in [...]\typo3\sysext\extbase\Classes\Persistence\Generic\Qom\QueryObjectModelFactory.php line 122
Function:
/**
* Test
*
* #param string $text
* #return Objects
*/
public function findWordsByText($text) {
$query = $this->createQuery();
$query->getQuerySettings()->setRespectStoragePage(FALSE);
$query->getQuerySettings()->setReturnRawQueryResult(TRUE);
$constraint1 = $query->like('title', $text . '%');
// $constraint1 = $query->statement("SELECT * FROM pages WHERE title LIKE '" . $text . "%'");
$constraint2 = $query->like('title', 'new');
$constraint = array($constraint1, $constraint2);
$query->matching(
$query->logicalOr($constraint)
);
$records = $query->execute();
return $records;
}
If I change follwing lines:
$query->matching(
$query->logicalOr($constraint)
);
To - logicalAnd:
$query->matching(
$query->logicalAnd($constraint)
);
...the query is empty.
I don't see your problem. You want to return all records that either contain the term $text in the title field or have 'new' in the title field. So just use a simple logicalOr query:
/**
* Test
*
* #param string $text
* #return Objects
*/
public function findWordsByText($text) {
$query = $this->createQuery();
$query->getQuerySettings()->setRespectStoragePage(FALSE);
$query->getQuerySettings()->setReturnRawQueryResult(TRUE);
$query->matching(
$query->logicalOr(
$query->like('title', '%' . $text . '%'),
$query->like('title', 'new')
)
);
return $query->execute();
}
$query->statement() is for raw SQL queries and cannot be combined with createQuery.
By the way, if $text comes from a POST or GET input, don't forget to sanitize the input.

Propel ORM - how do I specify tables to reverse engineer?

I have a database with a few dozen tables. My app deals with just one of those tables. I'd like propel to generate the schema just for that table. Is there a way I can specify the table(s) in build.properties?
A simple extension of Propel (code examples based on Propel-1.6.8) to allow ignore a custom set of tables (configured using a property):
Extend build-propel.xml:
<!-- Ignore Tables Feature -->
<taskdef
name="propel-schema-reverse-ignore"
classname="task.PropelSchemaReverseIgnoreTask" classpathRef="propelclasses"/>
Extend build.xml:
<target name="reverse-ignore" depends="configure">
<phing phingfile="build-propel.xml" target="reverse-ignore"/>
</target>
Extend PropelSchemaReverseTask:
class PropelSchemaReverseIgnoreTask extends PropelSchemaReverseTask {
/**
* Builds the model classes from the database schema.
* #return Database The built-out Database (with all tables, etc.)
*/
protected function buildModel()
{
// ...
// Loads Classname reverse.customParserClass if present
$customParserClassname = $config->getBuildProperty("reverseCustomParserClass");
if ($customParserClassname!=null) {
$this->log('Using custom parser class: '.$customParserClassname);
$parser = $config->getConfiguredSchemaParserForClassname($con, $customParserClassname);
} else {
$parser = $config->getConfiguredSchemaParser($con);
}
// ...
}
}
Add function to GeneratorConfig:
class GeneratorConfig implements GeneratorConfigInterface {
// ...
/**
* Ignore Tables Feature:
* Load a specific SchemaParser class
*
* #param PDO $con
* #param string $clazzName SchemaParser class to load
* #throws BuildException
* #return Ambigous <SchemaParser, unknown>
*/
public function getConfiguredSchemaParserForClassname(PDO $con = null, $clazzName)
{
$parser = new $clazzName();
if (!$parser instanceof SchemaParser) {
throw new BuildException("Specified platform class ($clazz) does implement SchemaParser interface.", $this->getLocation());
}
$parser->setConnection($con);
$parser->setMigrationTable($this->getBuildProperty('migrationTable'));
$parser->setGeneratorConfig($this);
return $parser;
}
}
Extend MysqlSchemaParser:
class MysqlSchemaIgnoreParser extends MysqlSchemaParser {
public function parse(Database $database, Task $task = null)
{
$this->addVendorInfo = $this->getGeneratorConfig()->getBuildProperty('addVendorInfo');
$stmt = $this->dbh->query("SHOW FULL TABLES");
// First load the tables (important that this happen before filling out details of tables)
$tables = array();
$configIgnoreTables = $this->getGeneratorConfig()->getBuildProperty("reverseIgnoreTables");
$tables_ignore_list = array();
$tables_ignore_list = explode(",", $configIgnoreTables);
if ($task) {
$task->log("Reverse Engineering Tables", Project::MSG_VERBOSE);
}
while ($row = $stmt->fetch(PDO::FETCH_NUM)) {
$name = $row[0];
$type = $row[1];
if (!in_array($name, $tables_ignore_list)) {
// ...
} else {
$task->log("Ignoring table: " .$name);
}
}
// ...
}
}
Add new (optional) properties to build.properties:
# Propel Reverse Custom Properties
propel.reverse.customParserClass=MysqlSchemaIgnoreParser
propel.reverse.ignoreTables = table1,table2
A solution could be to write your own schema parser that ignores certain tables. These tables to exclude could be read from a config file or so.
Take a look at the MysqlSchemaParser.php to get an idea of how such a parser works. You could extend such an existing parser and just override the parse() method. When tables are added, you would check them for exclusion/inclusion and only add them if they meet your subset criteria.
How to create custom propel tasks is described in the Propel cookbook.
Instead of invoking propel-gen reverse you would then invoke your customized reverse engineering task.
If done neatly, I'd find it worth being added as a contribution to the Propel project and you'd definitely deserve fame for that! :-)