I can mock Time.now with a great timecop gem.
Time.now
=> 2018-05-13 18:04:46 +0300
Timecop.travel(Time.parse('2018.03.12, 12:00'))
Time.now
=> 2018-03-12 12:00:04 +0300
TeacherVacation.first.ends_at
Thu, 15 Mar 2018 12:00:00 MSK +03:00
TeacherVacation.where('ends_at > ?', Time.now).count
1
But (obviously) this wouldn't work while using NOW() in a query:
TeacherVacation.where('ends_at > NOW()').count
0
Can I mock NOW() so that it would return the results for a certain time?
Timecop is a great gem! I would recommend using Timecop.freeze instead of traveling for your instance; you want to keep your tests deterministic.
As far as I could find, there doesn't seem to be a way to mock SQL's functions. Some languages like Postgres allow overloading functions, but you would still need a way to interject, and there doesn't seem to be a way to use environment variables in SQL.
A co-worker seemed to be certain you could actually drop system/language functions and make your own, but I was concerned about how to recover them after you do that. Trying to go that route sounds like a pain.
Solutions?
Here are a couple of "solutions" that I've come up with today while fighting this problem. Note: I don't really care for them to be honest, but if it gets tests in place ¯\_(ツ)_/¯ They at least offer a way to get things "working".
Unfortunately there's no snazzy gem to control the time in SQL. I imagine you would need something crazy like a plugin to the DB, a hack, a hook, a man in the middle, a container that you could trick SQL into thinking the system time was something else. None of those hack ideas would surely be portable/platform agnostic unfortunately either.
Apparently there are some ways to set time in a docker container, but that sounds like a painful overhead for local testing, and doesn't fit the granularity of a per-test time to be set.
Another thing to note, for me we're running large complex raw SQL queries, so that's why it's important that when I run the SQL file for a test I can have proper dates, otherwise I would just be doing it through activerecord like you mentioned.
String Interpolation
I ran across this in some large queries that were being ran.
This definitely helps if you need to push some environment variables through, and you can inject your own "current_date" if you want. This would help too if you needed to utilize a certain time across multiple queries.
my_query.rb
<<~HEREDOC
SELECT *
FROM #{#prefix}.my_table
WHERE date < #{#current_date} - INTERVAL '5 DAYS'
HEREDOC
sql_runner.rb
class SqlRunner
def initialize(file_path)
#file_path = file_path
#prefix = ENV['table_prefix']
#current_date = Date.today
end
def run
execute(eval(File.read #file_path))
end
private
def execute(sql)
ActiveRecord::Base.connection.execute(sql)
end
end
The Dirty Update
The idea is to update the value from ruby land pushing your "time-copped" time into the database to overwrite the value generated by the SQL DB. You may need to get creative with your update for times, like querying for a time greater than a given time that doesn't target your timecop time that you'll be updating rows to.
The reason I don't care for this method is because it ends up feeling like you're just testing activerecord's functionality since you're not relying on the DB to set values it should be setting. You may have computations in your SQL that you're then recreating in the test to set some value to the right date, and then you're no longer doing the computation in the SQL so then you're not even actually testing it.
large_insert.sql
INSERT INTO some_table (
name,
created_on
)
SELECT
name,
current_date
FROM projects
JOIN people ON projects.id = people.project_id
insert_spec.rb
describe 'insert_test.sql' do
ACTUAL_DATE = Date.today
LARGE_INSERT_SQL = File.read('sql/large_insert.sql')
before do
Timecop.freeze Date.new(2018, 10, 28)
end
after do
Timecop.return
end
context 'populated same_table' do
before do
execute(LARGE_INSERT_SQL)
mock_current_dates(ACTUAL_DATE)
end
it 'has the right date' do
expect(SomeTable.last.created_on).to eq(Date.parse('2018.10.28')
end
end
def execute(sql_command)
ActiveRecord::Base.connection.execute(sql_command)
end
def mock_current_dates(actual_date)
rows = SomeTable.where(created_on: actual_date)
# Use our timecop datetime
rows.update_all(created_on: Date.today)
end
Fun Caveat: specs wrap in their own transactions (you can turn that off, but it's a nice feature) so if your SQL has a transaction in it, you'll need to write code to remove it for the specs, or have your runner wrap your code in transactions if you need them. They'll run but then your SQL will kill off the spec transaction and you'll have a bad time. You can create a spec/support to help out with this if you go the route of cleaning up during tests, if I were in a newer project I would go with writing a runner that wraps the queries in transactions if you need them -- even though this isn't evident in the SQL files #abstraction.
Maybe there's something out there that lets you set your system time, but that sounds terrifying modifying your system's actual time.
I think the solution for this is DI (dependency injection)
def NOW(time = Time.now)
time
end
In test
current_test = Time.new(2018, 5, 13)
p NOW(current_test)
In production
p NOW
I'm trying to find a simple yet robust way to convert time between arbitrary time zones.
This: http://www.cpearson.com/excel/TimeZoneAndDaylightTime.aspx explains only how to convert betwen my (current) TZ and another TZ.
Those two SO articles (Getting Windows Time Zone Information (C++/MFC) and How do you get info for an arbitrary time zone in Windows?) talk about getting the information from the registry.
That sounds a bit too convoluted and time-consuming; moreover, it appears that Windows stores TZs in their "full names" (such as (UTC-08:00) Pacific Time (US & Canada)) and I'd rather refer to TZs using abbreviations (such as EDT). Moreover, relying on Windows registry could also be unsafe: different users might have different versions and some might not be up to date. That would mean a report run by two persons might provide two different results!
Is there a simpler way that will also be robust? Writing a lookup table could work for some time but then it will be broken when a government decides to abolish DST or change anything else.
Maybe get a list of TZs from Internet and parse it? Would that be safe enough?
Update 1
I've made my research and explored the possibilities, but this problem is not as trivial as it might seem. If you think that the function shall look like bTime = aTime + 3, then please reconsider. Timezones and DSTs are in a state of constant flux.
Read this for reference: list of pending / proposed timezone changes. Note that some countries are actually changing their timezones, not just DST settings! And Brazil changed the date on which they change their clocks to winter time! A static lookup table would be broken very quickly by all those changes.
Update 2
I'm not looking into a quick and dirty hack, I can come up with that myself. I'm not wanting to write something and forget about it; I'd like to create a function once that could be safely used by other people for different internal projects without the maintenance nightmare. Hard-coding constants that are known to change once in a while is a very bad software design (think Y2K bug caused by a very, very old piece of code).
Update 3
This database looks good (although I'm not sure if it's stable enough): https://timezonedb.com/api. They even have a TZ conversion call - exactly what I need! I will probably try to parse XML from VBA and share my results.
The API at https://timezonedb.com/references/convert-time-zone is indeed a great place to get the correct worldwide time, timezone, and timezone-offset between two locations, taking into account past/future Daylight Savings changes.
A problem with your suggested method of specifying only the Time Zone Abbreviations (such as "convert PST to EST") is that this API takes your zones literally, even if they are incorrect.
So, if Toronto is currently on EDT but you specify EST, you'll probably get the incorrect time. Using "full names" like (UTC-08:00) Pacific Time (US & Canada) would have the same issue.
A way around that is to specify the time zone names like America/Vancouver (as listed here), or else specify the city, country and/or region name with the appropriate parameters.
I wrote a function to figure it out but it only applies to certain countries (see further down).
What time was it in Toronto last Halloween at 11:11pm Vancouver time?
http://api.timezonedb.com/v2/convert-time-zone?key=94RKE4SAXH67&from=America/Vancouver&to=America/Toronto&time=1509516660
Result: (Default is XML but JSON is also available.)
<result>
<status>OK</status>
<message/>
<fromZoneName>America/Vancouver</fromZoneName>
<fromAbbreviation>PDT</fromAbbreviation>
<fromTimestamp>1509516660</fromTimestamp>
<toZoneName>America/Toronto</toZoneName>
<toAbbreviation>EDT</toAbbreviation>
<toTimestamp>1509527460</toTimestamp>
<offset>10800</offset>
</result>
Getting the data programmatically:
There are plenty of options and lookup methods you will have to decide upon, but here's one example using a VBA Function:
What will be the time difference between Vancouver & Berlin on Christmas Day?
Input Time: 2018-12-25 00:00:00 = Vancouver Local Unix time 1545724800
Function GetTimeZoneOffsetHours(fromZone As String, _
toZone As String, UnixTime As Long) As Single
Const key = "94RKE4SAXH67"
Const returnField = "<offset>"
Dim HTML As String, URL As String
Dim XML As String, pStart As Long, pStop As Long
URL = "http://api.timezonedb.com/v2/convert-time-zone?key=" & key & _
"&from=" & fromZone & "&to=" & toZone & "&time=" & UnixTime
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", URL, False
.Send
XML = .ResponseText
End With
pStart = InStr(XML, returnField)
If pStart = 0 Then
MsgBox "Something went wrong!"
Exit Function
End If
pStart = pStart + Len(returnField) + 1
pStop = InStr(pStart, XML, "</") - 1
GetTimeZoneOffsetHours = Val(Mid(XML, pStart, pStop - pStart)) / 60
End Function
Sub testTZ()
Debug.Print "Time Zone Offset (Vancouver to Berlin) = " & _
GetTimeZoneOffsetHours("America/Vancouver", _
"Europe/Berlin", 1545724800) & " hours"
End Sub
Unix/UTC Timestamps:
Unix time is defined as "the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970."
You can convert times between Unix and/or UTC or Local time at: epochconverter.com ... the site also has conversion formulas for several programming languages.
For example, the formua to convert Unix time to GMT/UTC in Excel is:
=(A1 / 86400) + 25569
You could also download static files (in SQL or CSV format) here instead of caling the API, and the page also has sample queries. However use caution: it's easier to make mistakes with Daylight Savings (as mentioned above).
I made a dummy account to get the "demo" used in the examples, but you should get your own (free) key for long-term use. (I'm not responsible if it gets locked out from over-use!)
An good alternative Time Zone API is Google Maps Time Zone API. The difference is that you specify Latitude & Longitude. It seems to work just fine without a key You'll need to register for a key.
What will the Time Zone Offset be on June 1st at the White House?
https://maps.googleapis.com/maps/api/timezone/json?location=38.8976,-77.0365×tamp=1527811200&key={YourKeyHere}
Result:
{
"dstOffset" : 0,
"rawOffset" : -18000,
"status" : "OK",
"timeZoneId" : "America/Toronto",
"timeZoneName" : "Eastern Standard Time"
}
The Offset will be -18000 seconds (-5 hours).
Determining when Daylight Savings is in effect
Below is a function I put together so I could "trust" the Daylight Savings (DST) values I was getting from a different API, however (as discussed by others) the rules have no pattern plus are constantly changing country by country, even town by town in some parts of the world, so this only will work in countries where:
DST begins on the Second Sunday of March every year
DST end on the First Sunday of November every year
The applicable countries are Bahamas, Bermuda, Canada, Cuba, Haiti, St. Pierre & United States. (Source: Daylight saving time by country**)
Function IsDST(dateTime As Date) As Boolean
'Returns TRUE if Daylight Savings is in effect during the [dateTime]
'DST Start (adjust clocks forward) Second Sunday March at 02:00am
'DST end (adjust clocks backward) First Sunday November at 02:00am
Dim DSTStart As Date, DSTstop As Date
DSTStart = DateSerial(Year(dateTime), 3, _
(14 - Weekday(DateSerial(Year(dateTime), 3, 1), 3))) + (2 / 24)
DSTstop = DateSerial(Year(dateTime), 11, _
(7 - Weekday(DateSerial(Year(dateTime), 11, 1), 3))) + (2 / 24)
IsDST = (dateTime >= DSTStart) And (dateTime < DSTstop)
End Function
And a couple examples of how I could use function IsDST*:
Public Function UTCtoPST(utcDateTime As Date) As Date
'Example for 'PST' time zone, where Offset = -7 during DST, otherwise if -8
If IsDST(utcDateTime) Then
UTCtoPST = utcDateTime - (7 / 24)
Else
UTCtoPST = utcDateTime - (8 / 24)
End If
End Function
Function UTCtimestampMStoPST(ByVal ts As String) As Date
'Example for 'PST', to convert a UTC Unix Time Stamp to 'PST' Time Zone
UTCtimestampMStoPST = UTCtoPST((CLng(Left(ts, 10)) / 86400) + 25569)
End Function
* Note that function IsDST is incomplete: It does not take into account the hours just before/after IsDST takes actually effect at 2am. Specifically when, in spring, the clock jumps forward from the last instant of 01:59 standard time to 03:00 DST and that day has 23 hours, whereas in autumn the clock jumps backward from the last instant of 01:59 DST to 01:00 standard time, repeating that hour, and that day has 25 hours ...but, if someone wants to add that functionality to update the function, feel free! I was having trouble wrapping my head around that last part, and didn't immediately need that level of detail, but I'm sure others would appreciate it!
Finally one more alternative is an API that I use to for polling current/future/historical weather data for various purposes — and also happens to provide Timezone Offset — is DarkSky.
It queries by latitude/longitude and is free (up to 1000 calls/day) and provides "ultra-accurate weather data" (more-so in the USA, where it predicts weather down to the minute and to the square-yard! — but quite accurate I've seen for the unpredictable Canadian West Coast Canada!)
Response is in JSON only, and the very last line is Time Zone Offset versus UTC/GMT time.
DarkSky Sample Call:
https://api.darksky.net/forecast/85b57f827eb89bf903b3a796ef53733c/40.70893,-74.00662
It says it's supposed to rain for the next 60 hours at Stack Overflow's Head Office. ☂
...but I dunno, it looks like a pretty nice day so far! ☀
(flag)
Im afraid anything to do with timezones is never a simple task (ask any web designer and they will say it is a massive challenge)
there are 2 ways to solve your problem
1) The Easy way - Create a central list which all other workbooks are linked to. This can be saved on SharePoint or on a shared drive, then all you have to do is update this one table
2) The hard way - Use a website API to get the latest timezone data. https://www.amdoren.com/ is a good site, you can get a free API key by signing up. The only issue is you then have to parse the Json file from the website. This isn't easy but if you google "vba parse json" you will find some solutions (it generally requires importing some libraries and using other peoples code as a starting point)
Hope you find the right solution, and if you do might be worth sharing it as im sure there will be others with same issue.
I'm using Oracle Business Intelligence, and creating a custom report. I've been able to create and have everything work except there's an issue in using parameters on scheduled reports.
This is forcing me to hard code the auto-updating dates in the Data model. Is anyone able to assist with the best way to replace my parameters below?
Essentially, I'm looking for START_DATE_PARAMETER to be replaced with the System Date MINUS 15, and END_DATE_PARAMETER to be replaces with System Date PLUS 1.
AND table1.EFFECTIVE_START_DATE >= :START_DATE_PARAMETER
AND table1.EFFECTIVE_START_DATE <= :END_DATE_PARAMETER
AND table2.EFFECTIVE_END_DATE >= :START_DATE_PARAMETER
AND table2.EFFECTIVE_END_DATE <= :END_DATE_PARAMETER
I appreciate everyone's feedback. I am fairly new to SQL, and am trying to develop my skills so all of the input is much appreciated.
Because my parameters are meant to include more than I'll actually need, I can potentially use either the 'CURRENT_TIMESTAMP' or 'SYSDATE' solution. The one piece of the puzzle that I was missing was the inclusion of 'TRUNC('
AND table1.EFFECTIVE_START_DATE >= (TRUNC(CURRENT_TIMESTAMP) - 15)
AND table1.EFFECTIVE_START_DATE <= (TRUNC(CURRENT_TIMESTAMP) + 1)
AND table2.EFFECTIVE_END_DATE >= (TRUNC(CURRENT_TIMESTAMP) - 15)
AND table2.EFFECTIVE_END_DATE <= (TRUNC(CURRENT_TIMESTAMP) + 1)
Trying to get records that were created this year, I stumbled upon this great question. The second answer says you get all records from a model that were created today by saying:
Model.where("created_at >= ?", Time.now.beginning_of_day)
So, naturally, I tried the same thing with Time.now.beginning_of_year, and it works just fine.
However, what struck me as interesting is that the outputted query (I tried it in the console) is
SELECT COUNT(*) FROM `invoices` WHERE (created_at >= '2012-12-31 23:00:00')
I wasn't aware that 2013 already began at 2012-12-31 23:00:00? How's that?
If you haven't set it yet, you should set your timezone in the config/application.rb file. Look for the line that begins with config.time_zone. (If you aren't sure what value to give, you can run rake time:zones:all to get a list of all available timezones.)
Once you've set your timezone, you should use Time.zone.now, as opposed to Time.now. This will properly "scope" your times to your timezone.
Check the API for more details on TimeWithZone.
Generally speaking, the SQL queries that I write return unformatted data and I leave it to the presentation layer, a web page or a windows app, to format the data as required. Other people that I work with, including my boss, will insist that it is more efficient to have the database do it. I'm not sure that I buy that and believe that even if there was a measurable performance gain by having the database do it, that there are more compelling reasons to generally avoid this.
For example, I will place my queries in a Data Access layer with the intent of potentially reusing the queries whenever possible. Given this, I ascertain that the queries are more likely to be able to be reused if the data remains in their native type rather than converting the data to a string and applying formatting functions on them, for example, formatting a date column to a DD-MMM-YYYY format for display. Sure, if the SQL was returning the dates as formatted strings, you could reverse the process to revert the value back to a date data type, but this seems awkward, for lack of a better word. Furtehrmore, when it comes to formatting other data, for example, a machine serial number made up of a prefix, base and suffix with separating dashes and leading zeros removed in each sub field, you risk the possibility that you may not be able to correctly revert back to the original serial number when going in the other direction. Maybe this is a bad example, but I hope you see the direction I am going with this...
To take things a step further, I see people write VERY complex SQLs because they are essentially writing what I would call presentation logic into a SQL instead of returning simple data and then applying this presentation logic in the presentation layer. In my mind, this results in very complex, difficult to maintain and more brittle SQL that is less adaptable to change.
Take the following real-life example of what I found in our system and tell me what you think. The rational I was given for this approach was that this made the web app very simple to render the page as it used the following 1-line snippet of classic ADO logic in a Classic ASP web app to process the rows returned:
oRS.GetString ( , , "</td>" & vbCrLf & "<td style=""font-size:x-small"" nowrap>" ,"</td>" & vbCrLf & "</tr>" & vbCrLf & "<tr>" & vbCrLf & _
"<td style=""font-size:x-small"" nowrap>" ," " ) & "</td>" & vbCrLf & "</tr>" & vbCrLf & _
Here's the SQL itself. While I appreciate the author's ability to write a complex SQL, I feel like this is a maintenance nightmare. Am I nuts? The SQL is returning a list of programs that are current running against our database and the status of each:
Because the SQL did not display with CR/LFs when I pasted here, I decided to put the SQL on an otherwise empty personal Google site. Please feel free to comment. Thanks.
By the way-This SQL was actually constructed using VB Script nested WITHIN a classic ASP page, not calling a stored procedure, so you have the additional complexity of embedded concatentations and quoted markup, if you know what I mean, not to mention lack of formatting. The first thing I did when I was asked to help to debug the SQL was to add a debug.print of the SQL output and throw it through a SQL formatter that I just found. Some of the formatting was lost in pasting at the following link:
Edit(Andomar): copied inline: (external link removed, thanks-Chad)
SELECT
Substring(Datename("dw",start_datetime),1,3)
+ ', '
+ Cast(start_datetime AS VARCHAR) "Start Time (UTC/GMT)"
,program_name "Program Name"
,run_sequence "Run Sequence"
,CASE
WHEN batchno = 0
THEN Char(160)
WHEN batchno = NULL
THEN Char(160)
ELSE Cast(batchno AS VARCHAR)
END "Batch #" /* ,Replace(Replace(detail_log ,'K:\' ,'file://servernamehere/DiskVolK/') ,'\' ,'/') "log"*/ /* */
,Cast('<a href="GOIS_ViewLog.asp?Program_Name=' AS VARCHAR(99))
+ Cast(program_name AS VARCHAR)
+ Cast('&Run_Sequence=' AS VARCHAR)
+ Cast(run_sequence AS VARCHAR)
+ Cast('&Page=1' AS VARCHAR)
+ ''
+ Cast('">'
+ CASE
WHEN end_datetime >= start_datetime
THEN CASE
WHEN end_datetime <> 'Jan 1 1900 2:00 PM'
THEN CASE
WHEN (success_code = 10
OR success_code = 0)
AND exit_code = 10
THEN CASE
WHEN errorcount = 0
THEN 'Completed Successfully'
ELSE 'Completed with Errors'
END
WHEN success_code = 100
AND exit_code = 10
THEN 'Completed with Errors'
ELSE CASE
WHEN program_name <> 'FileDepCheck'
THEN 'Failed'
ELSE 'File not found'
END
END
ELSE CASE
WHEN success_code = 10
AND exit_code = 0
THEN 'Failed; Entries for Input File Missing'
ELSE 'Aborted'
END
END
ELSE CASE
WHEN ((Cast(Datediff(mi,start_datetime,Getdate()) AS INT) <= 240)
OR ((SELECT
Count(* )
FROM
MASTER.dbo.sysprocesses a(nolock)
INNER JOIN gcsdwdb.dbo.update_log b(nolock)
ON a.program_name = b.program_name
WHERE a.program_name = update_log.program_name
AND (Abs(Datediff(n,b.start_datetime,a.login_time))) < 1) > 0))
THEN 'Processing...'
ELSE 'Aborted without end date'
END
END
+ '</a>' AS VARCHAR) "Status / Log"
,Cast('<a href="' AS VARCHAR)
+ Replace(Replace(detail_log,'K:\','file://servernamehere/DiskVolK/'),
'\','/')
+ Cast('" title="Click to view Detail log text file"' AS VARCHAR(99))
+ Cast('style="font-family:comic sans ms; font-size:12; color:blue"><img src="images\DetailLog.bmp" border="0"></a>' AS VARCHAR(999))
+ Char(160)
+ Cast('<a href="' AS VARCHAR)
+ Replace(Replace(summary_log,'K:\','file://servernamehere/DiskVolK/'),
'\','/')
+ Cast('" title="Click to view Summary log text file"' AS VARCHAR(99))
+ Cast('style="font-family:comic sans ms; font-size:12; color:blue"><img src="images\SummaryLog.bmp" border="0"></a>' AS VARCHAR(999)) "Text Logs"
,errorcount "Error Count"
,warningcount "Warning Count"
,(totmsgcount
- errorcount
- warningcount) "Information Message Count"
,CASE
WHEN end_datetime > start_datetime
THEN CASE
WHEN Cast(Datepart("hh",(end_datetime
- start_datetime)) AS INT) > 0
THEN Cast(Datepart("hh",(end_datetime
- start_datetime)) AS VARCHAR)
+ ' hr '
ELSE ' '
END
+ CASE
WHEN Cast(Datepart("mi",(end_datetime
- start_datetime)) AS INT) > 0
THEN Cast(Datepart("mi",(end_datetime
- start_datetime)) AS VARCHAR)
+ ' min '
ELSE ' '
END
+ CASE
WHEN Cast(Datepart("ss",(end_datetime
- start_datetime)) AS INT) > 0
THEN Cast(Datepart("ss",(end_datetime
- start_datetime)) AS VARCHAR)
+ ' sec '
ELSE ' '
END
ELSE CASE
WHEN end_datetime = start_datetime
THEN '< 1 sec'
ELSE CASE
WHEN ((Cast(Datediff(mi,start_datetime,Getdate()) AS INT) <= 240)
OR ((SELECT
Count(* )
FROM
MASTER.dbo.sysprocesses a(nolock)
INNER JOIN gcsdwdb.dbo.update_log b(nolock)
ON a.program_name = b.program_name
WHERE a.program_name = update_log.program_name
AND (Abs(Datediff(n,b.start_datetime,a.login_time))) < 1) > 0))
THEN 'Running '
+ Cast(Datediff(mi,start_datetime,Getdate()) AS VARCHAR)
+ ' min'
ELSE ' '
END
END
END "Elapsed Time" /* ,end_datetime "End Time (UTC/GMT)" ,datepart("hh" ,
(end_datetime - start_datetime)) "Hr" ,datepart("mi" ,(end_datetime - start_datetime)) "Mins" ,datepart("ss" ,(end_datetime - start_datetime)) "Sec" ,datepart("ms" ,(end_datetime - start_datetime)) "mSecs" ,datepart("dw" ,start_datetime) "dp" ,case when datepart("dw" ,start_datetime) = 6 then ' Fri' when datepart("dw" ,start_datetime) = 5 then ' Thu' else '1' end */
,totalrows "Total Rows"
,inserted "Rows Inserted"
,updated "Rows Updated" /* ,success_code "succ" ,exit_code "exit" */
FROM
update_log
WHERE start_datetime >= '5/29/2009 16:15'
ORDER BY start_datetime DESC
The answer is obviously "just retrieve output". Formatting on the SQL server has the following problems:
it increases the network traffic from the SQL server
SQL has very poor string handling functionality
SQL servers are not optimised to perform string manipulation
you are using server CPU cycles which could better be used for query processing
it may make life difficult (or impossible) for the query optimiser
you have to write many more queries to support different formatting
you may have to write different queries to support formatting on different browsers
you can't re-use queries for different purposes
I'm sure there are many more.
SQL should not be formatting, period. It's a relational algebra for extracting (when using SELECT) data from the database.
Getting the DBMS to format the data for you is the wrong thing to do, and that should be left to your own code (outside the DBMS). The DBMS is generally under enough load as it is without having to do your presentation work for you. It's also optimized for data retrieval, not presentation.
I know DBAs that would call for my immediate execution if I tried to do something like that :-)
The concept of formatting output in SQL does sort of break the whole concept of seperation of presentation and data, not only that, but there are a number of conditions that might arise:
What if you need to localise your date formats? UK uses a different date format to the US,
for example - are you going into internationalize all the way back up to your data layer?
What if the rules of formatting change? I.e. Some text needs to be formatted in a different way to comply with some new corporate policy? Again, you would need to go all the way back to the data layer.
If we take a web context, how do you decide on escaping values? Different forms of escaping might be desired if you are outputting to a web page, or to JSON, or elsewhere...
Not only that, but SQL string manipulation functions are not typically very zippy.
I'm the developer responsible for the reporting engine of my company's product. In simple terms the engine works by building an XML document of the data to go into a report from the database, and then transforming the XML any which way to build a web-page, or a PDF or a Word document based on user requirement.
When I started five years ago I had the database formatting the output, although I'm pleased to say nothing I wrote was as horrific as the questions example. Over time I've moved the other way and now the XML holds only the raw data, and this is tidied up during the presentation.
Our software uses Traffic Lights as a quick at-a-glance status indicator, so we have a lot of char fields in the database storing 'R', 'A', 'G', 'U' to represent red, amber, green and unknown. I had several tricks such as SELECTS with embedded CASE statements to tranform single character codes into their English counterparts:
SELECT CASE status WHEN 'R' THEN 'Red' WHEN 'G' THEN 'Green' ...etc...
Sorting can't be done on the native codes; Users expect things to be in two orders: Red, Amber, Green or Green, Amber, Red; so I had corresponding SORT columns as well
SELECT
CASE status WHEN 'R' THEN 'Red' WHEN 'G' THEN 'Green' WHEN 'A' THEN 'Amber' END as status,
CASE status WHEN 'R' THEN 0 WHEN 'A' THEN 1 WHEN 'G' THEN 2 END as sort
FROM
table
ORDER BY
sort
That's just a brief example. I had other tricks for doing date formatting, assembly of names, etc.
This of course led to problems making the application multi-language since English is boiled into the database. I'd need to lookup a customer locale and write lots of multi-language CASES to support other languages. Not good. Also dates were a problem. Americans like their dates mm/dd and Europeans do dd/mm.
It also led to other duplication problems. If someone added a fourth or fifth traffic light option I have to modify all my SQL when the new status is already represented in code as a Java enum or something, that I could lookup once I'd read the single character from the database.
It became far, far easier in my case to just have the database return the raw data and for me to write a suite of Comparators and formatters to present the data in a document in the user's native language and encoding. If I was starting over again today that'd be what I'd do.
I think there's a place for some kinds of transforms on the way out of SQL, and it depends on the calling program's expectations.
For instance, if a datetime is appropriate, it should be returned natively. On the other hand, if you are only returning a year in a datetime field (or a quarter, like 1/1, 4/1, 7/1, 10/1), and the client is expected to parse out the information, put it in a separate column (like year = 2008 or quarter = '2008Q1'). Some code translations from code to description (dropping the code column and only emitting the description). There are reasonable cases where concatenation and string building are appropriate.
Your particular example is a place where it's inappropriate and while on the surface it looks like looser coupling (only change the SP in the database) it can actually create stronger coupling by forcing additional SPs to be written for different usages instead of multiple UIs being able to use the same SP. And then multiple SPs might need to be changed in sync as the system evolves.
When considering whether to format your data on behalf of your presentation layer, consider that your "presentation layer" may be a web service or other program. You may start by doing the formatting on behalf of a piece of UI code, only to later need the same query to be used by a web service, which will have different requirements.
A favorite of mine was a set of stored procedures which all formatted date/times. In the local timezone. It didn't work quite so well when called by a web service from a different timezone. It worked even less well when the regional settings of the database server changed, changing the date/time format. Oh, and it didn't work at midnight, since it truncated the "00:00" at the end.
OTOH, it was very convenient for the UI.
Most people I know disagree with me here, but I kinda like this approach. So I'll list some advantages:
SQL is very powerful: how many lines of C# would this query take?
SQL is very easy to update. I imagine this code is in a stored procedure, which you can change with a simple ALTER PROC. This can greatly reduce the time to roll in fixes.
SQL is fast; I've seen cases where introducing an ORM layer slowed down the application to a crawl.
SQL is easy to debug, and errors are easy to reproduce. Just run the query. Testing your fix is a question of running the new query.
SQL like this is not that hard to maintain, when it's properly formatted. There is not much SQL I can't understand in 5-10 minutes; but a multi-layered C# solutions can take a very long time, especially if you have to figure out which layer's abstraction is breaking.
I'm sure other people will list the disadvantages of the SQL approach.