PhpSpreadsheet import data from excel to database symfony 3.4 - symfony-3.4

i would like to know how can i use PhpSpreadsheet symfony 3 ,
should i use this bundle : roromix/SpreadsheetBundle,
can i have a little exemple how to use it to read rows from exemple_file.xlst
thank you

I would advice to use the PHPSpreadsheet package directly.
$spreadsheet = PhpSpreadsheet\IOFactory::load('exemple_file.xlst' );
$worksheet = $spreadsheet->getActiveSheet(); // get active worksheet
$rows = []; //empty array of rows
foreach ($worksheet->getRowIterator() AS $row) {
$cells = $row->getCellIterator();
$cells->setIterateOnlyExistingCells(FALSE); // iterates through all cells, including empty ones
$cellData = [];//
foreach ($cells as $cell) {
$cellData[] = $cell->getValue();
}
$rows[] = $cells;
}
This will create 2 - dimensional array "rows" containing all the worksheet data which you can then use to import to DB. Or you can import it directly row by row in the for loop.

Related

Workbook cell style in POI/NPOI doesn't work properly with multiple styles in workbook

I'm running into strange problem with .Net version of POI library for Excel Spreadsheets. I'm rewriting from text files to Excel 97-2003 documents and I'm like to add some formatting programmatically depend on some values gather at the begging of the program.
At the beginning, in the same method where I was creating a new cell from given value I was creating also a new Workbook CellStyle which was wrong, because I was running out of the styles very quickly (or I was just thought it was the cause of the problem).
Constructor of the class responsible for Excel Workbook:
public OldExcelWriter(TextWriter logger) : base(logger)
{
_workbook = new HSSFWorkbook();
_sheetData = _workbook.CreateSheet("sheet1");
_creationHelper = _workbook.GetCreationHelper();
}
Method that is calling all the chains of operations:
public void Write(string path, Data data)
{
FillSpreadSheetWithData(data, _sheetData);
SaveSpreadSheet(_workbook, path);
}
Long story short, in FillSpreadSheetWithData I have method for creating a row inside which I'm have a loop for each cell, so basically I'm iterating thru every column, passing IRow references to a row, column value, index and formatting information like this:
for (int j = 0; j < column.Count; j++)
{
CreateCell(row, column[j], j, data.Formatting[j]);
}
and while creating a new styles (for first shot I was trying to pass some date time values) I had situation like this in my rewrited Excel: screenshot of excel workbook
So formatting was passed correctly (also Horizontal Aligment etc.) but it get ugly after 15th row (always the same amount).
DateTime dataCell = DateTime.MaxValue;
var cell = row.CreateCell(columnIndex);
_cellStyle = _workbook.CreateCellStyle();
switch (format.Type)
{
case DataType.Date:
_cellStyle.DataFormat = _creationHelper.CreateDataFormat().GetFormat("m/dd/yyyy");
if (value.Replace("\n", "") != string.Empty)
{
dataCell = DateTime.ParseExact(value.Replace("\n", ""), "m/dd/yyyy",
System.Globalization.CultureInfo.InvariantCulture);
}
break;
}
switch (format.HorizontalAlignment)
{
case Enums.HorizontalAlignment.Left:
_cellStyle.Alignment = HorizontalAlignment.LEFT;
break;
case Enums.HorizontalAlignment.Center:
_cellStyle.Alignment = HorizontalAlignment.CENTER;
break;
}
if (dataCell != DateTime.MaxValue)
{
cell.CellStyle = _cellStyle;
cell.SetCellValue(dataCell);
dataCell = DateTime.MaxValue;
}
else
{
cell.CellStyle = _cellStyle;
cell.SetCellValue(value);
}
(It's not the cleanest code but I will don refactor after getting this work).
After running into this issue I thought that maybe I will create _cellStyle variable in the constructor and only change it's value depends on the case, because it's assigned to the new cell anyway and I see while debugging that object values are correct.
But after creating everything, it won't get any better. Styles was override by the last value of the style, and dates are spoiled also, but later: screnshoot of excel workbook after creating one instance of cell style
I'm running out of ideas, maybe I should create every combination of the cell styles (I'm using only few data formats and alignments) but before I will do that (because I'm running out of easy options right now) I wonder what you guys think that should be done here.
cell format is set to custom with date type
I am using this code to create my custom style and format. Its for XSSF Format of excel sheet. but it will work for HSSF format with some modification.
XSSFFont defaultFont = (XSSFFont)workbook.CreateFont();
defaultFont.FontHeightInPoints = (short)10;
defaultFont.FontName = "Arial";
defaultFont.Color = IndexedColors.Black.Index;
defaultFont.IsBold = false;
defaultFont.IsItalic = false;
XSSFCellStyle dateCellStyle = (XSSFCellStyle)workbook.CreateCellStyle();
XSSFDataFormat dateDataFormat = (XSSFDataFormat)workbook.CreateDataFormat();
dateCellStyle.SetDataFormat(dateDataFormat.GetFormat("m/d/yy h:mm")); //Replace format by m/dd/yyyy. try similar approach for phone number etc.
dateCellStyle.FillBackgroundColor = IndexedColors.LightYellow.Index;
//dateCellStyle.FillPattern = FillPattern.NoFill;
dateCellStyle.FillForegroundColor = IndexedColors.LightTurquoise.Index;
dateCellStyle.FillPattern = FillPattern.SolidForeground;
dateCellStyle.Alignment = HorizontalAlignment.Left;
dateCellStyle.VerticalAlignment = VerticalAlignment.Top;
dateCellStyle.BorderBottom = BorderStyle.Thin;
dateCellStyle.BorderTop = BorderStyle.Thin;
dateCellStyle.BorderLeft = BorderStyle.Thin;
dateCellStyle.BorderRight = BorderStyle.Thin;
dateCellStyle.SetFont(defaultFont);
//Apply your style to column
_sheetData.SetDefaultColumnStyle(columnIndex, dateCellStyle);
// Or you can also apply style cell wise like
var row = _sheetData.CreateRow(0);
for (int cellIndex = 0;cellIndex < TotalHeaderCount;cellIndex++)
{
row.Cells[cellIndex].CellStyle = dateCellStyle;
}

EPPlus - How to set data type of a column to General

In my ASP.NET Core 1.1 app I'm using EPPlus.Core to export data to Excel. Some columns exported have mostly numbers but rarely a text (e.g. N/A). These columns in generated Excel are showing (as expected) a green triangle on the top left corner of their cells. I want to get rid of those warnings.
Question: What's a good way of getting rid of those triangles when Excel is generated? I tried setting the format of these columns to text as follows but it did not work. I guess we need to set the format of these columns to General but I can't figure out how:
workSheet.Cells["D1:P1"].Style.Numberformat.Format = "General";
UPDATE:
Per a user's request, the code looks similar to following.
Error at inner loop for (var j = 4; j < testlist[i].Count(); j++){...}: MyViewModel does not contain a definition of Count()
Error at line if (testlist[i][j] is string){...}: cannot apply indexing with [] to an extension of type MyViewModel
Controller:
....
....
var testlist = (qry to load a MyViewModel).ToList();
using (ExcelPackage pkg= new ExcelPackage())
{
var ws = excel.Workbook.Worksheets.Add("TestWorkSheet");
ws.Cells[1, 1].LoadFromCollection(rc_excel, true);
//I'm starting from 2nd row and 5th column
for (var i = 1; i < testlist.Count; i++)
{
for (var j = 4; j < testlist[i].Count(); j++)
{
if (testlist[i][j] is string)
{
....
....
}
}
pkg.Save();
return(....);
}

Filter cached sqlJdbs query in Pentaho CE

I use sqlJdbs query as a data provider for my CCC controls. I use geospatial request in my query that's why I cache my results(Cache=True). Otherwise the request made long.
It works fine. However I have to use parameters in my query to filter resulting rows:
SELECT ...
FROM ...
WHERE someField IN (${aoi_param})
Is there some way to cache full set of rows and then apply WHERE to cached results without rebuilding new cache for each set of values in the ${aoi_param}?
What is the best practice?
So, I am not really sure that it is the best practice, but I solved my problem this way:
I included aoi_param to the Listeners and Parameters of my chart control
Then I filtered data set in Post Fetch:
function f(data){
var _aoi_param = this.dashboard.getParameterValue('${p:aoi_param}');
function isInArray(myValue, myArray) {
var arrayLength = myArray.length;
for (var i = 0; i < arrayLength; i++) {
if (myValue == myArray[i]) return true;
}
return false;
}
function getFiltered(cdaData, filterArray) {
var allCdaData = cdaData;
cdaData = {
metadata: allCdaData.metadata,
resultset: allCdaData.resultset.filter(function(row){
// 2nd column is an AOI id in my dataset
return isInArray(row[2], filterArray);
})
};
return cdaData;
}
var dataFiltered = getFiltered(data, _aoi_param);
return dataFiltered;
}
excluded WHERE someField IN (${aoi_param}) from the query of my sql over sqlJdbc component

How to get data from xlsx sheet using dataprovider in selenium TestNG?

Can someone give me the logic of how to retrieve data from an Excel sheet (latest Excel file format) using the data provider in Selenium?
I'm mostly looking for the for loop logic inside the data provider.
Bascially, TestNG Data Provider is anything you create.
If you want to read xlsx files you need to create a class which reads excel rows and returns dataset.
#DataProvider(name = "data")
public static Object[][] returnExcelSheetData()
throws BiffException, IOException
{
String absolutePath = filePath.concat("/").concat(fileName); //path to excel file
FileInputStream file = new FileInputStream(new File(absolutePath));
Workbook workbook = Workbook.getWorkbook(file);
Sheet worksheet = workbook.getSheet(sheetName); //sheet name
int ROWS = worksheet.getRows() - 1; //if headers are present - use -1
int COLS = worksheet.getColumns(); //read all columns
Object[][] dataset = new Object[ROWS][COLS];
for (int rowCount = 0; rowCount < ROWS; rowCount++) {
for (int colCount = 0; colCount < COLS; colCount++) {
dataset[rowCount][colCount] = worksheet.getCell(colCount,
rowCount + 1).getContents();
}
}
workbook.close();
file.close();
return dataset;
for loops interate through ALL columns and ALL rows and return data sets.
If you use only 1 row of data and 1 row of headers, DataProvider will pass data to test.
If you use 2 rows of data, method which invokes DataProvider will be invoked for each row (2 times for 2 rows of data)

Export SQL query data to Excel

I have a query that returns a very large data set. I cannot copy and paste it into Excel which I usually do. I have been doing some research on how to export directly to an Excel sheet. I am running SQL SERVER 2008 on a server running Microsoft Server 2003. I am trying to use the Microsoft.Jet.OLEDB.4.0 data provider and Excel 2007. I've pieced together a small piece of code that looks like this from what I've seen in examples.
INSERT INTO OPENDATASOURCE('Microsoft.Jet.OLEDB.4.0',
'Data Source=C:\Working\Book1.xlsx;Extended Properties=EXCEL 12.0;HDR=YES')
SELECT productid, price FROM dbo.product
However this is not working, I am getting an error message saying
"Incorrect syntax near the keyword 'SELECT'".
Does anyone have any ideas about how to do this or possibly a better approach?
I don't know if this is what you're looking for, but you can export the results to Excel like this:
In the results pane, click the top-left cell to highlight all the records, and then right-click the top-left cell and click "Save Results As". One of the export options is CSV.
You might give this a shot too:
INSERT INTO OPENROWSET
('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;Database=c:\Test.xls;','SELECT productid, price FROM dbo.product')
Lastly, you can look into using SSIS (replaced DTS) for data exports. Here is a link to a tutorial:
http://www.accelebrate.com/sql_training/ssis_2008_tutorial.htm
== Update #1 ==
To save the result as CSV file with column headers, one can follow the steps shown below:
Go to Tools->Options
Query Results->SQL Server->Results to Grid
Check “Include column headers when copying or saving results”
Click OK.
Note that the new settings won’t affect any existing Query tabs — you’ll need to open new ones and/or restart SSMS.
If you're just needing to export to excel, you can use the export data wizard.
Right click the database, Tasks->Export data.
I had a similar problem but with a twist - the solutions listed above worked when the resultset was from one query but in my situation, I had multiple individual select queries for which I needed results to be exported to Excel. Below is just an example to illustrate although I could do a name in clause...
select a,b from Table_A where name = 'x'
select a,b from Table_A where name = 'y'
select a,b from Table_A where name = 'z'
The wizard was letting me export the result from one query to excel but not all results from different queries in this case.
When I researched, I found that we could disable the results to grid and enable results to Text. So, press Ctrl + T, then execute all the statements. This should show the results as a text file in the output window. You can manipulate the text into a tab delimited format for you to import into Excel.
You could also press Ctrl + Shift + F to export the results to a file - it exports as a .rpt file that can be opened using a text editor and manipulated for excel import.
Hope this helps any others having a similar issue.
For anyone coming here looking for how to do this in C#, I have tried the following method and had success in dotnet core 2.0.3 and entity framework core 2.0.3
First create your model class.
public class User
{
public string Name { get; set; }
public int Address { get; set; }
public int ZIP { get; set; }
public string Gender { get; set; }
}
Then install EPPlus Nuget package. (I used version 4.0.5, probably will work for other versions as well.)
Install-Package EPPlus -Version 4.0.5
The create ExcelExportHelper class, which will contain the logic to convert dataset to Excel rows. This class do not have dependencies with your model class or dataset.
public class ExcelExportHelper
{
public static string ExcelContentType
{
get
{ return "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"; }
}
public static DataTable ListToDataTable<T>(List<T> data)
{
PropertyDescriptorCollection properties = TypeDescriptor.GetProperties(typeof(T));
DataTable dataTable = new DataTable();
for (int i = 0; i < properties.Count; i++)
{
PropertyDescriptor property = properties[i];
dataTable.Columns.Add(property.Name, Nullable.GetUnderlyingType(property.PropertyType) ?? property.PropertyType);
}
object[] values = new object[properties.Count];
foreach (T item in data)
{
for (int i = 0; i < values.Length; i++)
{
values[i] = properties[i].GetValue(item);
}
dataTable.Rows.Add(values);
}
return dataTable;
}
public static byte[] ExportExcel(DataTable dataTable, string heading = "", bool showSrNo = false, params string[] columnsToTake)
{
byte[] result = null;
using (ExcelPackage package = new ExcelPackage())
{
ExcelWorksheet workSheet = package.Workbook.Worksheets.Add(String.Format("{0} Data", heading));
int startRowFrom = String.IsNullOrEmpty(heading) ? 1 : 3;
if (showSrNo)
{
DataColumn dataColumn = dataTable.Columns.Add("#", typeof(int));
dataColumn.SetOrdinal(0);
int index = 1;
foreach (DataRow item in dataTable.Rows)
{
item[0] = index;
index++;
}
}
// add the content into the Excel file
workSheet.Cells["A" + startRowFrom].LoadFromDataTable(dataTable, true);
// autofit width of cells with small content
int columnIndex = 1;
foreach (DataColumn column in dataTable.Columns)
{
int maxLength;
ExcelRange columnCells = workSheet.Cells[workSheet.Dimension.Start.Row, columnIndex, workSheet.Dimension.End.Row, columnIndex];
try
{
maxLength = columnCells.Max(cell => cell.Value.ToString().Count());
}
catch (Exception) //nishanc
{
maxLength = columnCells.Max(cell => (cell.Value +"").ToString().Length);
}
//workSheet.Column(columnIndex).AutoFit();
if (maxLength < 150)
{
//workSheet.Column(columnIndex).AutoFit();
}
columnIndex++;
}
// format header - bold, yellow on black
using (ExcelRange r = workSheet.Cells[startRowFrom, 1, startRowFrom, dataTable.Columns.Count])
{
r.Style.Font.Color.SetColor(System.Drawing.Color.White);
r.Style.Font.Bold = true;
r.Style.Fill.PatternType = OfficeOpenXml.Style.ExcelFillStyle.Solid;
r.Style.Fill.BackgroundColor.SetColor(Color.Brown);
}
// format cells - add borders
using (ExcelRange r = workSheet.Cells[startRowFrom + 1, 1, startRowFrom + dataTable.Rows.Count, dataTable.Columns.Count])
{
r.Style.Border.Top.Style = ExcelBorderStyle.Thin;
r.Style.Border.Bottom.Style = ExcelBorderStyle.Thin;
r.Style.Border.Left.Style = ExcelBorderStyle.Thin;
r.Style.Border.Right.Style = ExcelBorderStyle.Thin;
r.Style.Border.Top.Color.SetColor(System.Drawing.Color.Black);
r.Style.Border.Bottom.Color.SetColor(System.Drawing.Color.Black);
r.Style.Border.Left.Color.SetColor(System.Drawing.Color.Black);
r.Style.Border.Right.Color.SetColor(System.Drawing.Color.Black);
}
// removed ignored columns
for (int i = dataTable.Columns.Count - 1; i >= 0; i--)
{
if (i == 0 && showSrNo)
{
continue;
}
if (!columnsToTake.Contains(dataTable.Columns[i].ColumnName))
{
workSheet.DeleteColumn(i + 1);
}
}
if (!String.IsNullOrEmpty(heading))
{
workSheet.Cells["A1"].Value = heading;
// workSheet.Cells["A1"].Style.Font.Size = 20;
workSheet.InsertColumn(1, 1);
workSheet.InsertRow(1, 1);
workSheet.Column(1).Width = 10;
}
result = package.GetAsByteArray();
}
return result;
}
public static byte[] ExportExcel<T>(List<T> data, string Heading = "", bool showSlno = false, params string[] ColumnsToTake)
{
return ExportExcel(ListToDataTable<T>(data), Heading, showSlno, ColumnsToTake);
}
}
Now add this method where you want to generate the excel file, probably for a method in the controller. You can pass parameters for your stored procedure as well. Note that the return type of the method is FileContentResult. Whatever query you execute, important thing is you must have the results in a List.
[HttpPost]
public async Task<FileContentResult> Create([Bind("Id,StartDate,EndDate")] GetReport getReport)
{
DateTime startDate = getReport.StartDate;
DateTime endDate = getReport.EndDate;
// call the stored procedure and store dataset in a List.
List<User> users = _context.Reports.FromSql("exec dbo.SP_GetEmpReport #start={0}, #end={1}", startDate, endDate).ToList();
//set custome column names
string[] columns = { "Name", "Address", "ZIP", "Gender"};
byte[] filecontent = ExcelExportHelper.ExportExcel(users, "Users", true, columns);
// set file name.
return File(filecontent, ExcelExportHelper.ExcelContentType, "Report.xlsx");
}
More details can be found here
I see that you’re trying to export SQL data to Excel to avoid copy-pasting your very large data set into Excel.
You might be interested in learning how to export SQL data to Excel and update the export automatically (with any SQL database: MySQL, Microsoft SQL Server, PostgreSQL).
To export data from SQL to Excel, you need to follow 2 steps:
Step 1: Connect Excel to your SQL database‍ (Microsoft SQL Server, MySQL, PostgreSQL...)
Step 2: Import your SQL data into Excel
The result will be the list of tables you want to query data from your SQL database into Excel:

Step1: Connect Excel to an external data source: your SQL database
Install An ODBC
Install A Driver
Avoid A Common Error
Create a DSN
Step 2: Import your SQL data into Excel
Click Where You Want Your Pivot Table
Click Insert
Click Pivot Table
Click Use an external data source, then Choose Connection
Click on the System DSN tab
Select the DSN created in ODBC Manager
Fill the requested username and password
Avoid a Common Error
Access The Microsoft Query Dialog Box
Click on the arrow to see the list of tables in your database
Select the table you want to query data from your SQL database into Excel
Click on Return Data when you’re done with your selection
To update the export automatically, there are 2 additional steps:
Create a Pivot Table with an external SQL data source
Automate Your SQL Data Update In Excel With The GETPIVOTDATA Function
I’ve created a step-by-step tutorial about this whole process, from connecting Excel to SQL, up to having the whole thing automatically updated. You might find the detailed explanations and screenshots useful.