October 2012 - Echobase-commits - mailman.cloud.codelutin.com

r731 - trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions
by tchemit＠users.forge.codelutin.com Oct. 17, 2012

Oct. 17, 2012

Author: tchemit Date: 2012-10-17 17:16:03 +0200 (Wed, 17 Oct 2012) New Revision: 731 Url: http://forge.codelutin.com/repositories/revision/echobase/731 Log: fixes #1602: Impossible de faire une recherche sur Operation.id Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/AbstractJSONPaginedAction.java Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/AbstractJSONPaginedAction.java =================================================================== --- trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/AbstractJSONPaginedAction.java 2012-10-17 12:11:49 UTC (rev 730) +++ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/AbstractJSONPaginedAction.java 2012-10-17 15:16:03 UTC (rev 731) @@ -23,6 +23,7 @@ */ package fr.ifremer.echobase.ui.actions; +import org.apache.commons.collections.CollectionUtils; import org.apache.commons.lang3.StringUtils; import org.apache.struts2.json.JSONException; import org.apache.struts2.json.JSONUtil; @@ -155,6 +156,14 @@ Collection<Map<String, String>> rules = (Collection<Map<String, String>>) filterObject.get("rules"); + if (CollectionUtils.isNotEmpty(rules)) { + for (Map<String, String> rule : rules) { + //TODO tchemit-2012-10-17 Fix this bug + if ("ID".equals(rule.get("field"))) { + rule.put("field", "id"); + } + } + } pager = builder. setFilterOperationGroup(groupOp). addRules(rules).

1 0

r730 - trunk/src/site/rst
by tchemit＠users.forge.codelutin.com Oct. 17, 2012

Oct. 17, 2012

Author: tchemit Date: 2012-10-17 14:11:49 +0200 (Wed, 17 Oct 2012) New Revision: 730 Url: http://forge.codelutin.com/repositories/revision/echobase/730 Log: >From scmwebeditor -- Modified: trunk/src/site/rst/index.rst Modified: trunk/src/site/rst/index.rst =================================================================== --- trunk/src/site/rst/index.rst 2012-10-17 12:11:25 UTC (rev 729) +++ trunk/src/site/rst/index.rst 2012-10-17 12:11:49 UTC (rev 730) @@ -27,7 +27,7 @@ ======== Présentation -------------- +------------ L'Institut Français pour l'Exploration de la Mer (Ifremer) fournit une suite de logiciels libres pour stocker des données acoustiques halieutiques et calculer des indicateurs sur la base de données de campagnes. L'objectif est de contribuer à la

1 0

r729 - trunk/src/site/rst
by tchemit＠users.forge.codelutin.com Oct. 17, 2012

Oct. 17, 2012

Author: tchemit Date: 2012-10-17 14:11:25 +0200 (Wed, 17 Oct 2012) New Revision: 729 Url: http://forge.codelutin.com/repositories/revision/echobase/729 Log: >From scmwebeditor -- Modified: trunk/src/site/rst/index.rst Modified: trunk/src/site/rst/index.rst =================================================================== --- trunk/src/site/rst/index.rst 2012-10-12 08:45:11 UTC (rev 728) +++ trunk/src/site/rst/index.rst 2012-10-17 12:11:25 UTC (rev 729) @@ -27,7 +27,7 @@ ======== Présentation ------------- +------------- L'Institut Français pour l'Exploration de la Mer (Ifremer) fournit une suite de logiciels libres pour stocker des données acoustiques halieutiques et calculer des indicateurs sur la base de données de campagnes. L'objectif est de contribuer à la

1 0

r728 - trunk/echobase-tools
by tchemit＠users.forge.codelutin.com Oct. 12, 2012

Oct. 12, 2012

Author: tchemit Date: 2012-10-12 10:45:11 +0200 (Fri, 12 Oct 2012) New Revision: 728 Url: http://forge.codelutin.com/repositories/revision/echobase/728 Log: fix tests resources Modified: trunk/echobase-tools/pom.xml Modified: trunk/echobase-tools/pom.xml =================================================================== --- trunk/echobase-tools/pom.xml 2012-10-12 08:44:58 UTC (rev 727) +++ trunk/echobase-tools/pom.xml 2012-10-12 08:45:11 UTC (rev 728) @@ -35,7 +35,7 @@ <testResource> <directory>src/test/echobase-${csvVersion}</directory> <includes> - <include>**/*.csv</include> + <include>**/*.csv.gz</include> </includes> </testResource> <testResource>

1 0

r727 - trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders
by tchemit＠users.forge.codelutin.com Oct. 12, 2012

Oct. 12, 2012

Author: tchemit Date: 2012-10-12 10:44:58 +0200 (Fri, 12 Oct 2012) New Revision: 727 Url: http://forge.codelutin.com/repositories/revision/echobase/727 Log: fix baracouda location Modified: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/AbstractLoaderIT.java Modified: trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/AbstractLoaderIT.java =================================================================== --- trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/AbstractLoaderIT.java 2012-10-11 22:31:21 UTC (rev 726) +++ trunk/echobase-tools/src/test/java/fr/ifremer/echobase/tools/loaders/AbstractLoaderIT.java 2012-10-12 08:44:58 UTC (rev 727) @@ -90,7 +90,6 @@ msAccessFile = FileUtil.getFileFromPaths(TestHelper.getBasedir(), "src", "test", - "private", "baracouda.mdb" ); if (!msAccessFile.exists()) {

1 0

r726 - trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders
by tchemit＠users.forge.codelutin.com Oct. 11, 2012

Oct. 11, 2012

Author: tchemit Date: 2012-10-12 00:31:21 +0200 (Fri, 12 Oct 2012) New Revision: 726 Url: http://forge.codelutin.com/repositories/revision/echobase/726 Log: make Loader works with gz files Modified: trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/EntityLoader.java Modified: trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/EntityLoader.java =================================================================== --- trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/EntityLoader.java 2012-10-11 22:30:14 UTC (rev 725) +++ trunk/echobase-tools/src/main/java/fr/ifremer/echobase/tools/loaders/EntityLoader.java 2012-10-11 22:31:21 UTC (rev 726) @@ -38,7 +38,6 @@ import fr.ifremer.echobase.services.EchoBaseServiceSupport; import fr.ifremer.echobase.services.exportdb.ExportService; import fr.ifremer.echobase.services.importdb.EchoBaseImportModelFactory; -import fr.ifremer.echobase.services.importdb.ImportService; import fr.ifremer.echobase.tools.FileType; import org.apache.commons.io.IOUtils; import org.apache.commons.logging.Log; @@ -71,6 +70,7 @@ import java.util.List; import java.util.Map; import java.util.Set; +import java.util.zip.GZIPInputStream; /** * Support for Csv file generator. @@ -333,8 +333,8 @@ public void copyStreamToFile(File file, FileType fileType) throws IOException { String path = "/csv/" + fileType.name().toLowerCase() + "/" + - file.getName(); - InputStream stream = getClass().getResourceAsStream(path); + file.getName() + ".gz"; + InputStream stream = new GZIPInputStream(getClass().getResourceAsStream(path)); try { Preconditions.checkNotNull(stream, "could not find class-path resource " + path); @@ -345,9 +345,7 @@ outStream.close(); } } finally { - if (stream != null) { - stream.close(); - } + stream.close(); } }

1 0

r725 - trunk/echobase-tools/src/test/echobase-2012-04-12/csv/baracouda
by tchemit＠users.forge.codelutin.com Oct. 11, 2012

Oct. 11, 2012

Author: tchemit Date: 2012-10-12 00:30:14 +0200 (Fri, 12 Oct 2012) New Revision: 725 Url: http://forge.codelutin.com/repositories/revision/echobase/725 Log: refs #1592: Probl?\195?\168me de format de date sur les cell-data refs #1593: Probl?\195?\168me de format des positions pour les donn?\195?\169es de type localisation Modified: trunk/echobase-tools/src/test/echobase-2012-04-12/csv/baracouda/Cell.csv.gz trunk/echobase-tools/src/test/echobase-2012-04-12/csv/baracouda/Data.csv.gz Modified: trunk/echobase-tools/src/test/echobase-2012-04-12/csv/baracouda/Cell.csv.gz =================================================================== (Binary files differ) Modified: trunk/echobase-tools/src/test/echobase-2012-04-12/csv/baracouda/Data.csv.gz =================================================================== (Binary files differ)

1 0

r724 - in trunk: echobase-services/src/main/java/fr/ifremer/echobase/services/embeddedapplication echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery echobase-ui/src/main/resources/i18n echobase-ui/src/main/webapp/WEB-INF/jsp/exportQuery
by tchemit＠users.forge.codelutin.com Oct. 11, 2012

Oct. 11, 2012

Author: tchemit Date: 2012-10-11 19:13:23 +0200 (Thu, 11 Oct 2012) New Revision: 724 Url: http://forge.codelutin.com/repositories/revision/echobase/724 Log: fixes #1595: Permettre d'utiliser des requ?\195?\170tes imrbiqu?\195?\169es Added: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryInvalidNameException.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryNotFoundException.java Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/embeddedapplication/EmbeddedApplicationService.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryService.java trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/DownloadExportQueryResult.java trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/ExportQueryResult.java trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/GetExportQueryResult.java trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/ManageExportQuery.java trunk/echobase-ui/src/main/resources/i18n/echobase-ui_en_GB.properties trunk/echobase-ui/src/main/resources/i18n/echobase-ui_fr_FR.properties trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/exportQuery/exportQuery.jsp trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/exportQuery/exportQueryResult.jsp Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/embeddedapplication/EmbeddedApplicationService.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/embeddedapplication/EmbeddedApplicationService.java 2012-10-11 17:13:02 UTC (rev 723) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/embeddedapplication/EmbeddedApplicationService.java 2012-10-11 17:13:23 UTC (rev 724) @@ -40,6 +40,7 @@ import fr.ifremer.echobase.services.exportdb.ExportDbConfiguration; import fr.ifremer.echobase.services.exportdb.ExportDbMode; import fr.ifremer.echobase.services.exportdb.ExportDbService; +import fr.ifremer.echobase.services.exportquery.ExportQueryInvalidNameException; import fr.ifremer.echobase.services.exportquery.ExportQueryNameAlreadyExistException; import fr.ifremer.echobase.services.exportquery.ExportQueryService; import fr.ifremer.echobase.services.importdata.ImportException; @@ -144,7 +145,7 @@ protected EchoBaseUser importInternalDb(EmbeddedApplicationConfiguration model, File dir, - EchoBaseServiceContext newServiceContext) throws TopiaException, ExportQueryNameAlreadyExistException, WorkingDbConfigurationAlreadyExistException { + EchoBaseServiceContext newServiceContext) throws TopiaException, ExportQueryNameAlreadyExistException, WorkingDbConfigurationAlreadyExistException, ExportQueryInvalidNameException { File internalDir = new File(dir, "internaldb"); Added: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryInvalidNameException.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryInvalidNameException.java (rev 0) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryInvalidNameException.java 2012-10-11 17:13:23 UTC (rev 724) @@ -0,0 +1,35 @@ +package fr.ifremer.echobase.services.exportquery; + +/* + * #%L + * EchoBase :: Services + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 - 2012 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ + +/** + * To be thrown if a query has not a valid query name. + * + * @author tchemit <chemit(a)codelutin.com> + * @since 1.4 + */ +public class ExportQueryInvalidNameException extends Exception { + + private static final long serialVersionUID = 1L; +} \ No newline at end of file Property changes on: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryInvalidNameException.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Added: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryNotFoundException.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryNotFoundException.java (rev 0) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryNotFoundException.java 2012-10-11 17:13:23 UTC (rev 724) @@ -0,0 +1,35 @@ +package fr.ifremer.echobase.services.exportquery; + +/* + * #%L + * EchoBase :: Services + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 - 2012 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ + +/** + * When a export query is not found. + * + * @author tchemit <chemit(a)codelutin.com> + * @since 1.4 + */ +public class ExportQueryNotFoundException extends Exception { + + private static final long serialVersionUID = 1L; +} \ No newline at end of file Property changes on: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryNotFoundException.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryService.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryService.java 2012-10-11 17:13:02 UTC (rev 723) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/exportquery/ExportQueryService.java 2012-10-11 17:13:23 UTC (rev 724) @@ -32,6 +32,7 @@ import org.apache.commons.lang3.StringUtils; import org.nuiton.topia.TopiaException; import org.nuiton.topia.persistence.pager.TopiaPagerBean; +import org.nuiton.util.RecursiveProperties; import org.nuiton.util.csv.Export; import java.util.List; @@ -45,8 +46,12 @@ */ public class ExportQueryService extends EchoBaseServiceSupport { - public ExportQuery getExportQuery(String topiaId) { + public ExportQuery getExportQuery(String topiaId) throws ExportQueryNotFoundException { ExportQuery exportQuery = getEntityById(ExportQuery.class, topiaId); + + if (exportQuery == null) { + throw new ExportQueryNotFoundException(); + } ExportQuery entityToSave = newExportQuery(); entityToSave.setTopiaId(exportQuery.getTopiaId()); entityToSave.setName(exportQuery.getName()); @@ -58,7 +63,7 @@ } public ExportQuery createOrUpdate(ExportQuery exportQuery, - EchoBaseUser user) throws ExportQueryNameAlreadyExistException { + EchoBaseUser user) throws ExportQueryNameAlreadyExistException, ExportQueryInvalidNameException { try { ExportQueryDAO dao = getDAO(); @@ -70,6 +75,12 @@ String queryName = exportQuery.getName(); + if (!StringUtils.isAlphanumeric(queryName)) { + + // can not accept a non alpha numeric name + throw new ExportQueryInvalidNameException(); + } + // check query does not already exists with this name if (mustCreate) { @@ -117,15 +128,42 @@ } } + public String getSqlQuery(String queryId) throws ExportQueryNotFoundException { + + ExportQuery exportQuery = getExportQuery(queryId); + + String result = getSafeSql(exportQuery.getSqlQuery()); + return result; + } + + protected String getSafeSql(String sql) { + RecursiveProperties sqls = new RecursiveProperties(); + List<ExportQuery> queries = getEntities(ExportQuery.class); + for (ExportQuery query : queries) { + sqls.put(query.getName(), query.getSqlQuery()); + } + if (!sqls.containsKey(sql)) { + + sqls.put(sql, sql); + } + + String result = sqls.getProperty(sql); + return result; + } + public void testSql(String sql) throws TopiaException { - GenericSQLQuery sqlQuery = new GenericSQLQuery(sql, null); + String safeSql = getSafeSql(sql); + GenericSQLQuery sqlQuery = new GenericSQLQuery(safeSql, null); sqlQuery.getColumnNames(getTransaction()); } - public Map<String, Object>[] executeSql(String sql, TopiaPagerBean pager) { + public Map<String, Object>[] executeExportquery(String queryId, + TopiaPagerBean pager) throws ExportQueryNotFoundException { + String sql = getSqlQuery(queryId); + // get a query to count all rows for the request GenericSQLQuery sqlQuery = new GenericSQLQuery(sql, pager); try { @@ -160,8 +198,10 @@ } } - public String createCsvFileContent(String sql) { + public String createCsvFileContent(String queryId) throws ExportQueryNotFoundException { + String sql = getSqlQuery(queryId); + GenericSQLQuery sqlQuery = new GenericSQLQuery(sql, null); List<Map<String, Object>> rows; try { @@ -196,8 +236,6 @@ } protected ExportQueryDAO getDAO() { - return getDAO(ExportQuery.class, - ExportQueryDAO.class); + return getDAO(ExportQuery.class, ExportQueryDAO.class); } - } Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/DownloadExportQueryResult.java =================================================================== --- trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/DownloadExportQueryResult.java 2012-10-11 17:13:02 UTC (rev 723) +++ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/DownloadExportQueryResult.java 2012-10-11 17:13:23 UTC (rev 724) @@ -23,6 +23,7 @@ */ package fr.ifremer.echobase.ui.actions.exportQuery; +import fr.ifremer.echobase.entities.ExportQuery; import fr.ifremer.echobase.services.exportquery.ExportQueryService; import fr.ifremer.echobase.ui.actions.EchoBaseActionSupport; @@ -39,8 +40,13 @@ private static final long serialVersionUID = 1L; - /** Sql request to execute. */ - protected String sql; + /** + * Id of the query to call. + * + * @see ExportQuery#getTopiaId() + * @since 1.4 + */ + protected String queryId; /** Input stream of the file to download. */ protected transient InputStream inputStream; @@ -54,8 +60,8 @@ /** Content type of the file to download. */ protected String contentType; - public void setSql(String sql) { - this.sql = sql; + public void setQueryId(String queryId) { + this.queryId = queryId; } public void setFileName(String fileName) { @@ -84,7 +90,7 @@ ExportQueryService service = getService(ExportQueryService.class); contentType = "text/csv"; - String content = service.createCsvFileContent(sql); + String content = service.createCsvFileContent(queryId); contentLength = content.length(); inputStream = new ByteArrayInputStream(content.getBytes()); Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/ExportQueryResult.java =================================================================== --- trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/ExportQueryResult.java 2012-10-11 17:13:02 UTC (rev 723) +++ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/ExportQueryResult.java 2012-10-11 17:13:23 UTC (rev 724) @@ -24,6 +24,7 @@ package fr.ifremer.echobase.ui.actions.exportQuery; import fr.ifremer.echobase.EchoBaseTechnicalException; +import fr.ifremer.echobase.entities.ExportQuery; import fr.ifremer.echobase.services.exportquery.ExportQueryService; import fr.ifremer.echobase.ui.actions.EchoBaseActionSupport; import org.apache.commons.logging.Log; @@ -42,8 +43,13 @@ /** Logger. */ private static final Log log = LogFactory.getLog(ExportQueryResult.class); - /** Sql request to execute. */ - protected String sql; + /** + * Id of the query to call. + * + * @see ExportQuery#getTopiaId() + * @since 1.4 + */ + protected String queryId; /** Default file name for export. */ protected String fileName; @@ -51,10 +57,6 @@ /** names of columns from the executed sql request. */ protected String[] columnNames; - public String getSql() { - return sql; - } - public String[] getColumnNames() { return columnNames; } @@ -63,17 +65,23 @@ return fileName; } - public void setSql(String sql) { - this.sql = sql; + public String getQueryId() { + return queryId; } + public void setQueryId(String queryId) { + this.queryId = queryId; + } + @Override public String execute() throws Exception { // obtain columNames from the request ExportQueryService service = getService(ExportQueryService.class); + String sql = service.getSqlQuery(queryId); try { + columnNames = service.getColumnNames(sql); } catch (EchoBaseTechnicalException e) { //TODO add a real nice message Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/GetExportQueryResult.java =================================================================== --- trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/GetExportQueryResult.java 2012-10-11 17:13:02 UTC (rev 723) +++ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/GetExportQueryResult.java 2012-10-11 17:13:23 UTC (rev 724) @@ -23,6 +23,7 @@ */ package fr.ifremer.echobase.ui.actions.exportQuery; +import fr.ifremer.echobase.entities.ExportQuery; import fr.ifremer.echobase.services.exportquery.ExportQueryService; import fr.ifremer.echobase.ui.actions.AbstractJSONPaginedAction; @@ -38,14 +39,19 @@ private static final long serialVersionUID = 1L; - /** Sql request to execute. */ - protected String sql; + /** + * Id of the query to call. + * + * @see ExportQuery#getTopiaId() + * @since 1.4 + */ + protected String queryId; /** Datas of the given table. */ protected Map<?, ?>[] datas; - public void setSql(String sql) { - this.sql = sql; + public void setQueryId(String queryId) { + this.queryId = queryId; } public Map<?, ?>[] getDatas() { @@ -55,7 +61,8 @@ @Override public String execute() throws Exception { ExportQueryService service = getService(ExportQueryService.class); - datas = service.executeSql(sql, pager); + + datas = service.executeExportquery(queryId, pager); return SUCCESS; } } Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/ManageExportQuery.java =================================================================== --- trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/ManageExportQuery.java 2012-10-11 17:13:02 UTC (rev 723) +++ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/actions/exportQuery/ManageExportQuery.java 2012-10-11 17:13:23 UTC (rev 724) @@ -26,6 +26,7 @@ import com.opensymphony.xwork2.Preparable; import fr.ifremer.echobase.entities.EchoBaseUser; import fr.ifremer.echobase.entities.ExportQuery; +import fr.ifremer.echobase.services.exportquery.ExportQueryInvalidNameException; import fr.ifremer.echobase.services.exportquery.ExportQueryNameAlreadyExistException; import fr.ifremer.echobase.services.exportquery.ExportQueryService; import fr.ifremer.echobase.ui.actions.EchoBaseActionSupport; @@ -114,19 +115,20 @@ String result = INPUT; - boolean safeSql = checkQuery(getQuery().getSqlQuery()); + boolean safeSql = checkQuery(); if (safeSql) { try { - query = getService().createOrUpdate(getQuery(), - getEchoBaseSession().getUser() - ); + query = getService().createOrUpdate( + getQuery(), getEchoBaseSession().getUser()); result = SUCCESS; - } catch (ExportQueryNameAlreadyExistException e) { + } catch (ExportQueryInvalidNameException e) { addFieldError("query.name", + _("echobase.error.query.invalid.name")); + }catch (ExportQueryNameAlreadyExistException e) { + addFieldError("query.name", _("echobase.error.query.name.already.exists")); - } } return result; @@ -161,7 +163,7 @@ if (StringUtils.isNotEmpty(getQuery().getTopiaId())) { // test query - checkQuery(getQuery().getSqlQuery()); + checkQuery(); } return SUCCESS; } @@ -211,10 +213,11 @@ } } - private boolean checkQuery(String sql) { + private boolean checkQuery() { + boolean result; try { - getService().testSql(sql); + getService().testSql(getQuery().getSqlQuery()); result = true; } catch (Exception e) { Throwable cause = e.getCause(); Modified: trunk/echobase-ui/src/main/resources/i18n/echobase-ui_en_GB.properties =================================================================== --- trunk/echobase-ui/src/main/resources/i18n/echobase-ui_en_GB.properties 2012-10-11 17:13:02 UTC (rev 723) +++ trunk/echobase-ui/src/main/resources/i18n/echobase-ui_en_GB.properties 2012-10-11 17:13:23 UTC (rev 724) @@ -211,6 +211,7 @@ echobase.error.no.embeddedApplication.exportFileFound=New portable database not found echobase.error.no.exportDb.configurationFound=Database export configuration not found echobase.error.no.exportDb.exportFileFound=Database export archive not found +echobase.error.query.invalid.name=\=Query names must only contains alphanumeric characters echobase.error.query.name.already.exists=Query name already exists echobase.error.required.email=E-mail address required echobase.error.required.password=Password is required Modified: trunk/echobase-ui/src/main/resources/i18n/echobase-ui_fr_FR.properties =================================================================== --- trunk/echobase-ui/src/main/resources/i18n/echobase-ui_fr_FR.properties 2012-10-11 17:13:02 UTC (rev 723) +++ trunk/echobase-ui/src/main/resources/i18n/echobase-ui_fr_FR.properties 2012-10-11 17:13:23 UTC (rev 724) @@ -211,6 +211,7 @@ echobase.error.no.embeddedApplication.exportFileFound=Application embarquée générée non trouvée echobase.error.no.exportDb.configurationFound=Configuration de l'export de la base non trouvée echobase.error.no.exportDb.exportFileFound=Archive d'export de la base introuvable +echobase.error.query.invalid.name=Le nom de la requête doit être constitué uniquement de caractères alphanumériques echobase.error.query.name.already.exists=Nom de requête déjà utilisé echobase.error.required.email=Courriel obligatoire echobase.error.required.password=Le mot de passe est obligatoire Modified: trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/exportQuery/exportQuery.jsp =================================================================== --- trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/exportQuery/exportQuery.jsp 2012-10-11 17:13:02 UTC (rev 723) +++ trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/exportQuery/exportQuery.jsp 2012-10-11 17:13:23 UTC (rev 724) @@ -190,7 +190,7 @@ $.ajax( { url:'<s:url action="exportQueryResult" namespace="/exportQuery"/>', - data:{sql:$('[name="query.sqlQuery"]').val()}, + data:{queryId:$('[name="query.topiaId"]').val()}, success:function (data, textStatus, jqXHR) { $('#resultGrid').html(data); } Modified: trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/exportQuery/exportQueryResult.jsp =================================================================== --- trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/exportQuery/exportQueryResult.jsp 2012-10-11 17:13:02 UTC (rev 723) +++ trunk/echobase-ui/src/main/webapp/WEB-INF/jsp/exportQuery/exportQueryResult.jsp 2012-10-11 17:13:23 UTC (rev 724) @@ -47,7 +47,7 @@ <s:else> <s:url id="loadUrl" action="getExportQueryResult" namespace="/exportQuery" escapeAmp="false"> - <s:param name="sql" value="%{sql}"/> + <s:param name="queryId" value="%{queryId}"/> </s:url> <s:form action="downloadExportQueryResult" namespace="/exportQuery"> @@ -55,7 +55,7 @@ <fieldset class="ui-corner-all"> <legend><s:text name="echobase.legend.sqlQuery.result"/></legend> - <s:hidden key="sql" label=""/> + <s:hidden key="queryId" label=""/> <s:textfield key="fileName" required="true" size="100" label="%{getText('echobase.label.exportFileName')}"/> <s:submit value="%{getText('echobase.action.exportSqlData')}" align="left"/>

1 0

r723 - in trunk: echobase-domain/src/main/java/fr/ifremer/echobase echobase-domain/src/main/java/fr/ifremer/echobase/entities/references echobase-services/src/main/java/fr/ifremer/echobase/services/importdata
by tchemit＠users.forge.codelutin.com Oct. 11, 2012

Oct. 11, 2012

Author: tchemit Date: 2012-10-11 19:13:02 +0200 (Thu, 11 Oct 2012) New Revision: 723 Url: http://forge.codelutin.com/repositories/revision/echobase/723 Log: fixes #1590: Am?\195?\169liorer l'import des donn?\195?\169es de p?\195?\170che Added: trunk/echobase-domain/src/main/java/fr/ifremer/echobase/entities/references/SampleTypeImpl.java Modified: trunk/echobase-domain/src/main/java/fr/ifremer/echobase/EchoBasePredicates.java trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/importdata/CatchesImportService.java Modified: trunk/echobase-domain/src/main/java/fr/ifremer/echobase/EchoBasePredicates.java =================================================================== --- trunk/echobase-domain/src/main/java/fr/ifremer/echobase/EchoBasePredicates.java 2012-10-11 12:20:26 UTC (rev 722) +++ trunk/echobase-domain/src/main/java/fr/ifremer/echobase/EchoBasePredicates.java 2012-10-11 17:13:02 UTC (rev 723) @@ -26,10 +26,15 @@ import com.google.common.collect.Sets; import fr.ifremer.echobase.entities.data.Cell; import fr.ifremer.echobase.entities.data.Echotype; +import fr.ifremer.echobase.entities.data.Operation; +import fr.ifremer.echobase.entities.data.Sample; import fr.ifremer.echobase.entities.references.AcousticInstrument; import fr.ifremer.echobase.entities.references.CellType; +import fr.ifremer.echobase.entities.references.SampleTypeImpl; import fr.ifremer.echobase.entities.references.SizeCategory; +import org.apache.commons.collections.CollectionUtils; +import java.util.Collection; import java.util.Set; /** @@ -100,6 +105,26 @@ } }; + public static final Predicate<Operation> OPERATION_WITH_TOTAL_OR_UNSORTED_SAMPLE = new Predicate<Operation>() { + @Override + public boolean apply(Operation input) { + Collection<Sample> samples = input.getSample(); + + Set<String> acceptedTypes = Sets.newHashSet(SampleTypeImpl.TOTAL_SAMPLE_TYPE, SampleTypeImpl.UNSORTED_SAMPLE_TYPE); + boolean result = false; + if (CollectionUtils.isNotEmpty(samples)) { + for (Sample sample : samples) { + String sampleType = sample.getSampleType().getName(); + if (acceptedTypes.contains(sampleType)) { + result = true; + break; + } + } + } + return result; + } + }; + public static Predicate<Echotype> newEchotypeByNamePredicate(String name) { return new EchotypeByNamePredicate(name); } Added: trunk/echobase-domain/src/main/java/fr/ifremer/echobase/entities/references/SampleTypeImpl.java =================================================================== --- trunk/echobase-domain/src/main/java/fr/ifremer/echobase/entities/references/SampleTypeImpl.java (rev 0) +++ trunk/echobase-domain/src/main/java/fr/ifremer/echobase/entities/references/SampleTypeImpl.java 2012-10-11 17:13:02 UTC (rev 723) @@ -0,0 +1,39 @@ +package fr.ifremer.echobase.entities.references; + +/* + * #%L + * EchoBase :: Domain + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 - 2012 Ifremer, Codelutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * #L% + */ + +public class SampleTypeImpl extends SampleTypeAbstract { + + private static final long serialVersionUID = 1L; + + public static final String TOTAL_SAMPLE_TYPE = "Total"; + + public static final String UNSORTED_SAMPLE_TYPE = "Unsorted"; + + public static final String SORTED_SAMPLE_TYPE = "Sorted"; + + public static final String SUB_SAMPLE_TYPE = "Subsample"; + + public static final String INDIVIDUAL_SAMPLE_TYPE = "Individual"; +} Property changes on: trunk/echobase-domain/src/main/java/fr/ifremer/echobase/entities/references/SampleTypeImpl.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Modified: trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/importdata/CatchesImportService.java =================================================================== --- trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/importdata/CatchesImportService.java 2012-10-11 12:20:26 UTC (rev 722) +++ trunk/echobase-services/src/main/java/fr/ifremer/echobase/services/importdata/CatchesImportService.java 2012-10-11 17:13:02 UTC (rev 723) @@ -24,6 +24,7 @@ package fr.ifremer.echobase.services.importdata; import com.google.common.base.Preconditions; +import com.google.common.collect.Collections2; import com.google.common.collect.Maps; import fr.ifremer.echobase.EchoBaseFunctions; import fr.ifremer.echobase.EchoBasePredicates; @@ -39,6 +40,7 @@ import fr.ifremer.echobase.entities.references.SampleDataTypeDAO; import fr.ifremer.echobase.entities.references.SampleType; import fr.ifremer.echobase.entities.references.SampleTypeDAO; +import fr.ifremer.echobase.entities.references.SampleTypeImpl; import fr.ifremer.echobase.entities.references.SexCategory; import fr.ifremer.echobase.entities.references.SizeCategory; import fr.ifremer.echobase.entities.references.Species; @@ -113,6 +115,14 @@ configuration.addResult(importResult); } + // build for each operation the one with at least one sample typed + // *total* or *unsorted* + + Collection<Operation> operationWithTotalOrUnsortedSample = + Collections2.filter( + operations, + EchoBasePredicates.OPERATION_WITH_TOTAL_OR_UNSORTED_SAMPLE); + inputFile = configuration.getSubSampleFile(); if (inputFile.hasFile()) { @@ -120,6 +130,7 @@ configuration, inputFile, operationMap, + operationWithTotalOrUnsortedSample, speciesMap, sizeCategoryMap ); @@ -133,6 +144,7 @@ configuration, inputFile, operationMap, + operationWithTotalOrUnsortedSample, speciesMap); configuration.addResult(importResult); @@ -165,11 +177,11 @@ SampleDataTypeDAO sampleDataTypeDAO = getDAO(SampleDataType.class, SampleDataTypeDAO.class); SpeciesCategoryDAO speciesCategoryDAO = getDAO(SpeciesCategory.class, SpeciesCategoryDAO.class); - SampleType sampleTypeTotal = findByProperties(sampleTypeDAO, SampleType.PROPERTY_NAME, "Total"); + SampleType sampleTypeTotal = findByProperties(sampleTypeDAO, SampleType.PROPERTY_NAME, SampleTypeImpl.TOTAL_SAMPLE_TYPE); Preconditions.checkNotNull(sampleTypeTotal); - SampleType sampleTypeUnsorted = findByProperties(sampleTypeDAO, SampleType.PROPERTY_NAME, "Unsorted"); + SampleType sampleTypeUnsorted = findByProperties(sampleTypeDAO, SampleType.PROPERTY_NAME, SampleTypeImpl.UNSORTED_SAMPLE_TYPE); Preconditions.checkNotNull(sampleTypeUnsorted); - SampleType sampleTypeSorted = findByProperties(sampleTypeDAO, SampleType.PROPERTY_NAME, "Sorted"); + SampleType sampleTypeSorted = findByProperties(sampleTypeDAO, SampleType.PROPERTY_NAME, SampleTypeImpl.SORTED_SAMPLE_TYPE); Preconditions.checkNotNull(sampleTypeSorted); SampleDataType sampleDataTypeMeanLength = findByProperties(sampleDataTypeDAO, SampleDataType.PROPERTY_NAME, "MeanLengthcm"); @@ -302,6 +314,7 @@ CatchesImportConfiguration configuration, InputFile inputFile, Map<String, Operation> operationMap, + Collection<Operation> operationWithTotalOrUnsortedSample, Map<String, Species> speciesMap, Map<String, SizeCategory> sizeCategoryMap ) throws ImportException { @@ -331,7 +344,7 @@ SampleDataTypeDAO sampleDataTypeDAO = getDAO(SampleDataType.class, SampleDataTypeDAO.class); SpeciesCategoryDAO speciesCategoryDAO = getDAO(SpeciesCategory.class, SpeciesCategoryDAO.class); - SampleType sampleTypeSubsample = findByProperties(sampleTypeDAO, SampleType.PROPERTY_NAME, "Subsample"); + SampleType sampleTypeSubsample = findByProperties(sampleTypeDAO, SampleType.PROPERTY_NAME, SampleTypeImpl.SUB_SAMPLE_TYPE); Preconditions.checkNotNull(sampleTypeSubsample); SampleDataType sampleDataTypeNumberAtLength = findByProperties(sampleDataTypeDAO, SampleDataType.PROPERTY_NAME, "NumberAtLength"); @@ -355,6 +368,11 @@ doFlushTransaction(++rowNumber, inputFile, configuration); Operation operation = row.getOperation(); + checkOperationWithTotalOrUnsortedSample( + rowNumber, + operationWithTotalOrUnsortedSample, + operation); + Species species = row.getSpecies(); SizeCategory sizeCategory = row.getSizeCategory(); SexCategory sexCategory = row.getSexCategory(); @@ -425,11 +443,23 @@ } } + private void checkOperationWithTotalOrUnsortedSample( + int rowNumber, + Collection<Operation> operationWithTotalOrUnsortedSample, + Operation operation + ) throws ImportException { + if (!operationWithTotalOrUnsortedSample.contains(operation)) { + + // can not accept this import + throw new ImportException("At line " + rowNumber + ", wants to add a subsample sample but operation " + operation.getId() + " has no sample of type *Total*, nor *Unsorted*"); + } + } + private EchoBaseCsvFileImportResult importBiometrySampleFile( CatchesImportConfiguration configuration, InputFile inputFile, Map<String, Operation> operationMap, - Map<String, Species> speciesMap + Collection<Operation> operationWithTotalOrUnsortedSample, Map<String, Species> speciesMap ) throws ImportException { if (log.isInfoEnabled()) { @@ -454,7 +484,7 @@ SampleTypeDAO sampleTypeDAO = getDAO(SampleType.class, SampleTypeDAO.class); SpeciesCategoryDAO speciesCategoryDAO = getDAO(SpeciesCategory.class, SpeciesCategoryDAO.class); - SampleType sampleTypeIndividual = findByProperties(sampleTypeDAO, SampleType.PROPERTY_NAME, "Individual"); + SampleType sampleTypeIndividual = findByProperties(sampleTypeDAO, SampleType.PROPERTY_NAME, SampleTypeImpl.INDIVIDUAL_SAMPLE_TYPE); Preconditions.checkNotNull(sampleTypeIndividual); Map<String, Sample> samples = Maps.newTreeMap(); @@ -469,8 +499,14 @@ for (BiometrySampleImportRow row : importer) { doFlushTransaction(++rowNumber, inputFile, configuration); + Operation operation = row.getOperation(); + checkOperationWithTotalOrUnsortedSample( + rowNumber, + operationWithTotalOrUnsortedSample, + operation); + Species species = row.getSpecies(); int numFish = row.getNumFish();

1 0

r722 - in trunk: echobase-ui/src/main/java/fr/ifremer/echobase/ui/interceptors src/doc/reunions
by tchemit＠users.forge.codelutin.com Oct. 11, 2012

Oct. 11, 2012

Author: tchemit Date: 2012-10-11 14:20:26 +0200 (Thu, 11 Oct 2012) New Revision: 722 Url: http://forge.codelutin.com/repositories/revision/echobase/722 Log: CR reunion + add missing properties Added: trunk/src/doc/reunions/reunion-2012-10-11.txt Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/interceptors/CheckWorkingDbSelected.java trunk/src/doc/reunions/reunion-2012-09-24.txt Modified: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/interceptors/CheckWorkingDbSelected.java =================================================================== --- trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/interceptors/CheckWorkingDbSelected.java 2012-10-11 02:54:43 UTC (rev 721) +++ trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/interceptors/CheckWorkingDbSelected.java 2012-10-11 12:20:26 UTC (rev 722) @@ -4,7 +4,7 @@ * #%L * EchoBase :: UI * $Id$ - * $HeadURL:$ + * $HeadURL$ * %% * Copyright (C) 2011 - 2012 Ifremer, Codelutin * %% Property changes on: trunk/echobase-ui/src/main/java/fr/ifremer/echobase/ui/interceptors/CheckWorkingDbSelected.java ___________________________________________________________________ Modified: svn:keywords - Author Date Id Revision + Author Date Id Revision HeadURL Property changes on: trunk/src/doc/reunions/reunion-2012-09-24.txt ___________________________________________________________________ Modified: svn:keywords - Author Date Id Revision + Author Date Id Revision HeadURL Added: trunk/src/doc/reunions/reunion-2012-10-11.txt =================================================================== --- trunk/src/doc/reunions/reunion-2012-10-11.txt (rev 0) +++ trunk/src/doc/reunions/reunion-2012-10-11.txt 2012-10-11 12:20:26 UTC (rev 722) @@ -0,0 +1,109 @@ +======== +Echobase +======== + +Jeudi 11 octobre 2012 (Ifremer) + +Présents : Mathieu et Tony + +But +=== + +- Lissage des données +- validation version 1.3 + +Import des données de pêche +=========================== + +pour importer des subsamples, vérifier qu'il y a au moins un sample de type +'total' ou 'unsorted' qui corresponde à l'opération à lier. + +sinon erreur + +idem pour l'import sample biometry + + +voir http://forge.codelutin.com/issues/1590 + +Lissage des données +=================== + +problème chalut N0528 + PELGAS2009 (données samples) +---------------------------------------------------- + +il manque des catches de type total, mathieu me refournit les fichiers qui vont bien... et je rer + +voir http://forge.codelutin.com/issues/1591 + +problème de format de date +-------------------------- + +les nom de cellules + les Data (pour les cellules de type elementaire) ne respecte pas le format de date + +exemple cellule : + +1983-04-24 07:03:00Z + +Il faut que je reprenne les Data change le format des dataValue et aussi les nom de cellules qui correspondent + +voir http://forge.codelutin.com/issues/1592 + +problème des positions +---------------------- + +Il faut transformer les data de type position via l'algorithme fournit par mathieu. + +voir http://forge.codelutin.com/issues/1593 + +perte de cellules esdu +---------------------- + +il en manque :( Mathieu me fournit des pistes (données manquantes) pour que je +puisse comprendre ce qui se passe de travers. + +voir http://forge.codelutin.com/issues/1594 + +Divers +====== + +base embarquée +-------------- + +Régler le problème d'un chemin avec espace pour démarrer l'application embarquée. + +http://forge.codelutin.com/issues/1596 + +export données +-------------- + +permettre d'inclure des sous-requetes avec le pattern ${nomRequete}. + +Cela permet de réutiliser des requetes déjà créées. + +Rajouter un check sur les noms de requetes pour quelles soient java identifier +compiliant. + + +voir http://forge.codelutin.com/issues/1595 + +Autre +----- + +- Ajouter répertoire svn pour script de migration ToEchobase + +A faire +======= + +- Mathieu + - documentation + tests sur lissage des données +- Tony + - faire les derniers tickets :) + - mettre à jour la doc d'installation + +Prochaine réunion +================= + +- Jeudi 18 octbore 2012 à 10h00 (si tout va bien on lance la VSR) + + + Property changes on: trunk/src/doc/reunions/reunion-2012-10-11.txt ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native

1 0