I have a column in my table to which I have added a Full Text Index. When I write a query to do contains on that particular column, the query gives the output of those records which matches with contain and which were added to the table before 3-4 hours and later. Those records which are added recently to the table don't show up in the output even though their text matches with the contains text.
Create table Table1 (Id int, Name varchar(20), Message varchar(1000), CreatedAt datetime)
Message is the column which has full text index.
Can someone please help me as to why is this behavior of SQL and what can I do to rectify this?
It sounds like your full text index isn't being populated after changes to the data. You should either set up automatic population of the index or perform manual population whenever you update the data.
More here: MSDN: Populate Full-Text Indexes
Related
I have a fact table that gets updated daily with customer time on app info from a third-party platform that we use, and the identifying number has a bit of text appended to it. So if the customer ID number is 123, this table is getting populated with something like ABC_123. I need to pull this info for a particular cohort of customers based on their ID numbers, so was planning to create a temp table with the customer ID number and the time on app, and drop the appended bit of text. I so far have not had luck finding a way to split the text in that column using the "_" as a delimiter, and I'm hesitant to use a wildcard. Any advice?
Seems like it would be better to add a PERSISTED computed column to the table. Then you have both the original data, and the one you want and you can INDEX the PERSISTED column too.
ALTER TABLE dbo.YourTable ADD GoodID AS CONVERT(int,STUFF(BadID, 1, CHARINDEX('_',BadID),'')) PERSISTED;
db<>fiddle
I have a database table which constantly gets updated. I am looking to query only the changes/additions that have been made on rows with a specific attribute in a column. e.g. get the rows which have been changed/added, the 'description' column of which is "xyz". My end goal is to copy these rows to another table in another database. Is this even possible? The reason for not just querying and overwriting the rows in the other database is to avoid inefficiency.
What I have tried so far?
I am able to select query on the table to get the rows but it gives me all the rows, not the ones that have been changed or recently added. If i add these rows to the table in the other database, the only option I have is to overwrite the rows.
Log table logs the changes in a table but I can't put additional filters in SQL which tells me which of these changes are associated with 'description' column as 'xyz'.
Write your update statements to make use of OUTPUT to capture the before and after values and log them to a table of your choice.
Here is a really simple example update example that uses output to store the RowID, before and after values for the ActivityType column:
DECLARE #MyTableVar table (
SummaryBefore nvarchar(max),
SummaryAfter nvarchar(max),
RowID int
);
update DBA.dbo.dtest set ActivityType = 3
OUTPUT deleted.ActivityType,
inserted.ActivityType,
inserted.RowID
INTO #MyTableVar
select * From #MyTableVar
You can do it two ways
Have new date fields/columns like update_time and/or create_time(Can be defaulted if needed). These fields will indicate the status of the record. You need to save your previous_run_time and then your select query will look for records with update_time/create_time greater than previous_run_time, and then you can move these records to the new DB.
Have CDC turned on the source table, which is available by default in SQL server and then move only those records that have been impacted.
in sql, in any given table, in a column named "name", wih data type as text
if there are ten entries, suppose an entry in the column is "rohit". i want to show all the entries in the name column after rohit. and i do not know the row id or id. can it be done??
select * from your_table where name > 'rohit'
but in general you should not treat text columns like that.
a database is more than a collection of tables.
think about how to organize your data, what defines a datarow.
maybe, beside their name, there is another thing how you would classify such a row? some things like "shall be displayed?" "is modified" "is active"?
so if you had a second column, say display of type int and your table looked like
CREATE TABLE MYDATA
NAME TEXT,
DISPLAY INT NOT NULL DEFAULT(1);
you could flag every row with 1 or 0 whether it should be displayed or not and then your query could look like
SELECT * FROM MYDATA WHERE DISPLAY=1 ORDER BY NAME
to get your list of values.
it's not much of a difference with ten rows, you don't even need indexes here, but if you build something bigger, say 10,000+ rows, you'd be surprised how slow that would become!
in general, TEXT columns are good to select and display, but should be avoided as a WHERE condition as much as you can. Use describing columns, preferrably int fields which can be indexed with extreme high efficiency and an application doesn't get slower even if the record size goes over 100k.
You can use "default" keyword for it.
CREATE TABLE Persons (
ID int NOT NULL,
name varchar(255) DEFAULT 'rohit'
);
I am trying to retrieve data from tickets that meet search matches. The relevant bits of data here are that a ticket has a name, and any number of comments.
Currently I'm matching a search against the ticket name like so:
JOIN freetexttable(Tickets,TIC_Name,'Test ') s1
ON TIC_PK = s1.[key]
Where the [key] from the full text catalog is equal to TIC_PK.
This works well for me, and gives me access to s1.rank, which is important for me to sort by.
Now my problem is that this method wont work for ticket searching, because the key in the comment catalog is the comment PK, an doesn't give me any information I can use to link to the ticket.
I'm very perplexed about how to go about searching multiple descriptions and still getting a meaningful rank.
I'm pretty knew to full-text search and might be missing something obvious.
Heres my current attempt at getting what I need:
WHERE TIC_PK IN(
SELECT DES_TIC_FK FROM freetexttable(TicketDescriptions, DES_Description,'Test Query') as t
join TicketDescriptions a on t.[key] = a.DES_PK
GROUP BY DES_TIC_FK
)
This gets me tickets with comments that match the search, but I dont think it's possible to sort by the rank data freetexttable returns with this method.
To search the name and comments at the same time and get the most meaningful rank you should put all of this info into the same table -- a new table -- populated from your existing tables via an ETL process.
The new table could look something like this:
CREATE TABLE TicketsAndDescriptionsETL (
TIC_PK int,
TIC_Name varchar(100),
All_DES_Descriptions varchar(max),
PRIMARY KEY (TIC_PK)
)
GO
CREATE FULLTEXT INDEX ON TicketsAndDescriptionsETL (
TIC_Name LANGUAGE 'English',
All_DES_Descriptions LANGUAGE 'English'
)
Schedule this table to be populated either via a SQL job, triggers on the Tickets and TicketDescriptions tables, or some hook in your data layer. For tickets that have multiple TicketDescriptions records, combine the text of all of those comments into the All_DES_Descriptions column.
Then run your full text searches against this new table.
While this approach does add another cog to the machine, there's really no other way to perform full text searches across multiple tables and generate one rank.
I have a Fulltext index on one of my tables which contains some metadata and a document blob (PDF or Doc or RTF etc)
Sometimes there is an error indexing a row and therefore the row cannot be returned in Fulltext searches.
What query could I use to find out what items have NOT been indexed?
I thought something like this:
Select * from MyTable where MyTableID NOT IN
(
select MyTableID from MyTable
where contains(Title, Title)
)
And then work out which rows were not returned. But the inner query is not syntactically correct and I cant work it out.
Any ideas?
Cheers
Aaron
Bad news and good news:
Bad news - There is no way to find out what items have not been indexed just by using a simple query.
Good News - You can add a datetime on your fulltext table and store the insert date for each record on it. Then, you can create a Log table that will contains the last date that a population was executed. Using this table you can find out wich records were not indexed since last index population.
I dont know if I made myself clear. I just did what i said today. I created a job that will start a population, and another job that will check if the population is done and populate the log table with the last index population date.