- T3-commits - mailman.cloud.codelutin.com

[Git][ultreiaio/ird-t3][develop] improve Level 2 and 3 API + conversion API (See #263)
by Tony CHEMIT 22 Mar '18

22 Mar '18

Tony CHEMIT pushed to branch develop at ultreiaio / ird-t3 Commits: 5cacaf5b by Tony CHEMIT at 2018-03-22T15:18:39Z improve Level 2 and 3 API + conversion API (See #263) - - - - - 30 changed files: - t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ConvertSampleSetSpeciesFrequencyToWeightAction.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ConvertSetSpeciesFrequencyToWeightAction.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ExtrapolateSampleWeightToSetAction.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratum.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratum.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratumLoader.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2StratumConfiguration.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2StratumConfigurationIteratorWithCategories.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2StratumConfigurationIteratorWithoutCategories.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2Action.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2InputContext.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2OutputContext.java - − t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3ActionInputContext.java - − t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3ActionOutputContext.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3CatchStratum.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3SampleStratum.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3StratumConfiguration.java - − t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3StratumConfigurationIterator.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3StratumConfigurationIteratorWithCategories.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3StratumConfigurationIteratorWithoutCategories.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level3/Level3Action.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level3/Level3InputContext.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level3/Level3OutputContext.java - t3-actions/src/main/resources/ftl/fr/ird/t3/actions/data/level2/Level2Action.ftl - t3-actions/src/main/resources/ftl/fr/ird/t3/actions/data/level2/Level2Action_en.ftl - t3-actions/src/main/resources/ftl/fr/ird/t3/actions/data/level3/Level3Action.ftl - t3-actions/src/main/resources/ftl/fr/ird/t3/actions/data/level3/Level3Action_en.ftl - t3-actions/src/test/java/fr/ird/t3/actions/ActionResumeTest.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2ActionInputContext.java → t3-domain/src/main/java/fr/ird/t3/actions/stratum/LevelInputContext.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2ActionOutputContext.java → t3-domain/src/main/java/fr/ird/t3/actions/stratum/LevelOutputContext.java The diff was not included because it is too large. View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/5cacaf5b5a3b9965d6e88f946ba2f441… --- View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/5cacaf5b5a3b9965d6e88f946ba2f441… You're receiving this email because of your account on gitlab.com.

1 0

[Git][ultreiaio/ird-t3][develop] Great code refactor for level 2 and 3 (See #263)
by Tony CHEMIT 22 Mar '18

22 Mar '18

Tony CHEMIT pushed to branch develop at ultreiaio / ird-t3 Commits: 21196bfc by Tony CHEMIT at 2018-03-22T02:20:55Z Great code refactor for level 2 and 3 (See #263) - - - - - 13 changed files: - + t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2ActionInputContext.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2ActionOutputContext.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratum.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratumLoader.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratum.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2StratumConfiguration.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2StratumConfigurationIterator.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2Action.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3ActionInputContext.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3ActionOutputContext.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3CatchStratum.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3StratumConfiguration.java - + t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3StratumConfigurationIterator.java The diff was not included because it is too large. View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/21196bfcb99c0798d818bcd86fbda304… --- View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/21196bfcb99c0798d818bcd86fbda304… You're receiving this email because of your account on gitlab.com.

1 0

[Git][ultreiaio/ird-t3][develop] 2 commits: fix typo in i18n
by Tony CHEMIT 20 Mar '18

20 Mar '18

Tony CHEMIT pushed to branch develop at ultreiaio / ird-t3 Commits: c2d5de35 by Tony CHEMIT at 2018-03-20T15:51:19Z fix typo in i18n - - - - - 176ea4c3 by Tony CHEMIT at 2018-03-20T16:00:47Z [REFERENTIEL] Rajouter des champs de traduction (See #245) - fix dans la grille des maréés - - - - - 2 changed files: - t3-web/src/main/resources/i18n/t3-web_fr_FR.properties - t3-web/src/main/webapp/WEB-INF/jsp/admin/tripListGrid.jsp Changes: ===================================== t3-web/src/main/resources/i18n/t3-web_fr_FR.properties ===================================== --- a/t3-web/src/main/resources/i18n/t3-web_fr_FR.properties +++ b/t3-web/src/main/resources/i18n/t3-web_fr_FR.properties @@ -329,7 +329,7 @@ t3.level1.configuration.alreadyExecutedSteps=Etapes du niveau 1 déjà exécuté t3.level1.configuration.no.alreadyExecutedSteps=Aucune étape exécutée t3.level1.matching.sample.count=Nombre d'échantillons sélectionnés t3.level1.matching.trip.count=Nombre de marées sélectionnées -t3.level1.matching.tripAndSample.count=Nombre de marées / échantillons sélectionnés / rejectés +t3.level1.matching.tripAndSample.count=Nombre de marées / échantillons sélectionnés / rejetés t3.level1.not.matching.sample.count=Nombre d'échantillons rejetés t3.level2.inprogress=Traitement du niveau 2 en cours t3.level2.result=Résultat du traitement du niveau 2 ===================================== t3-web/src/main/webapp/WEB-INF/jsp/admin/tripListGrid.jsp ===================================== --- a/t3-web/src/main/webapp/WEB-INF/jsp/admin/tripListGrid.jsp +++ b/t3-web/src/main/webapp/WEB-INF/jsp/admin/tripListGrid.jsp @@ -71,13 +71,13 @@ deleteTrips: { title : '%{deleteKey}', icon: 'ui-icon-trash', topic: 'trips-rowDeleteTrips' } }"> <sjg:gridColumn name="id" title="id" hidden="true"/> - <sjg:gridColumn name="vessel.libelle" sortable="false" + <sjg:gridColumn name="vessel.label1" sortable="false" title='%{getText("t3.common.vessel")}'/> - <sjg:gridColumn name="vessel.fleetCountry.libelle" sortable="false" + <sjg:gridColumn name="vessel.fleetCountry.label1" sortable="false" title='%{getText("t3.common.fleetCountry")}'/> - <sjg:gridColumn name="vessel.flagCountry.libelle" sortable="false" + <sjg:gridColumn name="vessel.flagCountry.label1" sortable="false" title='%{getText("t3.common.flagCountry")}'/> <sjg:gridColumn name="departureDate" formatter="date" sortable="false" View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/compare/8a3cd0de840836f479078253c317306… --- View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/compare/8a3cd0de840836f479078253c317306… You're receiving this email because of your account on gitlab.com.

1 0

[Git][ultreiaio/ird-t3][develop] [N0.2][LOG] Petite bizarerie dans les logs (See #239)
by Tony CHEMIT 20 Mar '18

20 Mar '18

Tony CHEMIT pushed to branch develop at ultreiaio / ird-t3 Commits: 8a3cd0de by Tony CHEMIT at 2018-03-20T15:40:53Z [N0.2][LOG] Petite bizarerie dans les logs (See #239) - - - - - 1 changed file: - t3-actions/src/main/java/fr/ird/t3/actions/data/level0/ComputeRF2Action.java Changes: ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level0/ComputeRF2Action.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level0/ComputeRF2Action.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level0/ComputeRF2Action.java @@ -23,6 +23,7 @@ package fr.ird.t3.actions.data.level0; import com.google.common.collect.ArrayListMultimap; import com.google.common.collect.ListMultimap; import com.google.common.collect.Multimap; +import com.google.common.collect.TreeMultimap; import fr.ird.t3.entities.data.CompleteTrip; import fr.ird.t3.entities.data.RaisingFactor2; import fr.ird.t3.entities.data.RaisingFactor2TopiaDao; @@ -53,7 +54,6 @@ import java.util.HashSet; import java.util.Iterator; import java.util.List; import java.util.Set; -import java.util.TreeSet; import static org.nuiton.i18n.I18n.l; @@ -83,7 +83,7 @@ public class ComputeRF2Action extends AbstractLevel0Action<ComputeRF2Configurati private VesselSimpleTypeTopiaDao vesselSimpleTypeDAO; /** usable complete trips group by vessel */ private ListMultimap<Vessel, CompleteTrip> completeTripsByVessel; - private Set<String> tripStrDone; + private Multimap<String, String> tripDone; public ComputeRF2Action() { super(Level0Step.COMPUTE_RF2); @@ -119,7 +119,7 @@ public class ComputeRF2Action extends AbstractLevel0Action<ComputeRF2Configurati // use all fleets fleets = countryDAO.findAll(); } - tripStrDone = new TreeSet<>(); + tripDone = TreeMultimap.create(); List<Trip> tripList = getUsableTrips(landingHarbours, true); setTrips(tripList); completeTripsByVessel = ArrayListMultimap.create(); @@ -253,15 +253,17 @@ public class ComputeRF2Action extends AbstractLevel0Action<ComputeRF2Configurati // must update it raisingFactor2DAO.update(raisingFactor); } - if (rf2 != 1) { - nbTripsWithRF2 += trips.size(); - } // set rf2 to all trips for (CompleteTrip trip : trips) { String tripStr = decorate(trip); - String message = l(locale, "t3.level0.computeRF1.resume.rf2.for.trip", tripStr, rf2); - log.info(message); - addInfoMessage(message); + if (canLog(tripStr,"nbTrips")) { + if (rf2 != 1) { + nbTripsWithRF2++; + } + String message = l(locale, "t3.level0.computeRF1.resume.rf2.for.trip", tripStr, rf2); + log.info(message); + addInfoMessage(message); + } trip.applyRf2(rf2); markTripAsTreated(trip); } @@ -283,12 +285,12 @@ public class ComputeRF2Action extends AbstractLevel0Action<ComputeRF2Configurati tripCatchWeight = trip.getElementaryCatchTotalWeightRf1(species); } totalCatchWeight += tripCatchWeight; - if (tripStrDone.add(tripStr)) { + if (canLog(tripStr,"compute")) { addInfoMessage(l(locale, "t3.level0.computeRF2.resume.rf1.for.trip", tripStr, tripCatchWeight, tripLandingWeight)); } } else { // this trip does not have any landing catches for given species - if (tripStrDone.add(tripStr)) { + if (canLog(tripStr,"compute")) { addWarningMessage(l(locale, "t3.level0.computeRF2.resume.skip.for.trip", tripStr)); } } @@ -308,4 +310,12 @@ public class ComputeRF2Action extends AbstractLevel0Action<ComputeRF2Configurati } } + private boolean canLog(String key, String method) { + boolean exist = tripDone.containsEntry(key, method); + if (!exist) { + tripDone.put(key, method); + } + return !exist; + } + } View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/8a3cd0de840836f479078253c3173063… --- View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/8a3cd0de840836f479078253c3173063… You're receiving this email because of your account on gitlab.com.

1 0

[Git][ultreiaio/ird-t3][develop] 2 commits: [N2][N3] Gestion des catégories de poids -10/+10kg dans la stratification N2 et…
by Tony CHEMIT 20 Mar '18

20 Mar '18

Tony CHEMIT pushed to branch develop at ultreiaio / ird-t3 Commits: b6eea14f by Tony CHEMIT at 2018-03-20T14:46:29Z [N2][N3] Gestion des catégories de poids -10/+10kg dans la stratification N2 et N3 (See #263) - nettoyage de code et ajout de FIXME... - - - - - ff591952 by Tony CHEMIT at 2018-03-20T14:46:31Z [N1][N2][N3] Permettre la sélection mutliple d'océans (See #260) - correction sur le N1.5 (plus un peu de nettoyage de code) - - - - - 11 changed files: - t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ExtrapolateSampleWeightToSetAction.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratum.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratumLoader.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratum.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2Action.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3SampleStratum.java - t3-actions/src/test/java/fr/ird/t3/actions/ActionResumeTest.java - t3-domain/src/main/java/fr/ird/t3/actions/stratum/SampleStratum.java - t3-domain/src/main/java/fr/ird/t3/entities/reference/LengthWeightConversionHelper.java - t3-domain/src/main/java/fr/ird/t3/models/WeightCompositionAggregateModel.java - t3-domain/src/test/java/fr/ird/t3/entities/reference/LengthWeightConversionHelperTest.java Changes: ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ExtrapolateSampleWeightToSetAction.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ExtrapolateSampleWeightToSetAction.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ExtrapolateSampleWeightToSetAction.java @@ -38,7 +38,6 @@ import fr.ird.t3.entities.reference.Species; import fr.ird.t3.entities.reference.SpeciesTopiaDao; import fr.ird.t3.entities.reference.WeightCategories; import fr.ird.t3.entities.reference.WeightCategorySample; -import fr.ird.t3.entities.reference.WeightCategorySampleImpl; import fr.ird.t3.services.DecoratorService; import fr.ird.t3.services.ioc.InjectDAO; import org.apache.commons.logging.Log; @@ -46,6 +45,7 @@ import org.apache.commons.logging.LogFactory; import org.nuiton.util.TimeLog; import java.util.Collection; +import java.util.Date; import java.util.HashMap; import java.util.HashSet; import java.util.Map; @@ -54,7 +54,7 @@ import java.util.Set; import static org.nuiton.i18n.I18n.l; /** - * Extrapolate weigth from sample to their owing set. + * Extrapolate weight from sample to their owing set. * * @author Tony Chemit - dev(a)tchemit.fr * @since 1.0 @@ -69,16 +69,13 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { private LengthWeightConversionTopiaDao lengthWeightConversionDAO; @InjectDAO(entityType = SetSpeciesFrequency.class) private SetSpeciesFrequencyTopiaDao setSpeciesFrequencyDAO; - // @InjectDAO(entityType = WeightCategoryWellPlan.class) -// private WeightCategoryWellPlanTopiaDao weightCategoryWellPlanDAO; - // private WeightCategoryWellPlan categoryM10; -// private WeightCategoryWellPlan categoryP10; private LengthWeightConversionHelper conversionHelper; public ExtrapolateSampleWeightToSetAction() { super(Level1Step.EXTRAPOLATE_SAMPLE_WEIGHT_TO_SET); } + @SuppressWarnings("unused") public int getNbTreatedSets() { return nbTreatedSets; } @@ -87,6 +84,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { this.nbTreatedSets = nbTreatedSets; } + @SuppressWarnings("unused") public float getNbTreatedFishesInSamples() { return nbTreatedFishesInSamples; } @@ -95,6 +93,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { this.nbTreatedFishesInSamples = nbTreatedFishesInSamples; } + @SuppressWarnings("unused") public float getNbCreatedFishesInSetSpeciesFrequency() { return nbCreatedFishesInSetSpeciesFrequency; } @@ -106,10 +105,6 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { @Override protected void prepareAction() throws Exception { super.prepareAction(); -// // get the -10Kg category -// setCategoryM10(weightCategoryWellPlanDAO.forCodeEquals(1).findUnique()); -// // get the +10Kg category -// setCategoryP10(weightCategoryWellPlanDAO.forCodeEquals(2).findUnique()); conversionHelper = lengthWeightConversionDAO.newConversionHelper(); } @@ -137,21 +132,21 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { private void doExecuteTrip(Trip trip, Collection<Sample> samples) { incrementsProgression(); - //FIXME I replace ocean by trip.departureHarbour.ocean, check this is ok (See #260) - // get the length class +10kg limit - Map<Species, Integer> limitLengthClassBySpecie = getThredHoldPlus10ForSpecies(trip, samples, trip.getDepartureHarbour().getOcean()); Set<Activity> tripActivities = new HashSet<>(); // extrapolate for each sample set number to set for (Sample sample : samples) { long s0 = TimeLog.getTime(); incrementsProgression(); - String sampleStr = l(locale, "t3.level1.extrapolateSampleWeightToSet.sampleStr", - decorate(trip), sample.getSampleNumber()); + String sampleStr = l(locale, "t3.level1.extrapolateSampleWeightToSet.sampleStr", decorate(trip), sample.getSampleNumber()); float nb = sample.getTotalStandardiseSampleSpeciesFrequencyNumber(); addInfoMessage(l(locale, "t3.level1.extrapolateSampleWeightToSet.sample.nbFishes", sampleStr, nb)); nbTreatedFishesInSamples += nb; + Set<Species> species = SpeciesTopiaDao.getAllSpeciesFromSampleSpecies(samples); for (SampleSet sampleSet : sample.getSampleSet()) { Activity activity = sampleSet.getActivity(); + Ocean ocean = activity.getOcean(); + // get the length class +10kg limit + Map<Species, Integer> limitLengthClassBySpecie = getThresholdPlus10ForSpecies(species, ocean, activity.getRoute().getDate()); boolean added = tripActivities.add(activity); if (added) { // remove all stuff from activity @@ -240,7 +235,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { } /** - * Given a sample well, compute the {@link SampleWellSetWeight}, says : + * Given a sample well, compute the {@link SampleWellSetWeight}, says: * <ul> * <li>his total weight</<li> * <li>his weight of -10Kg category</<li> @@ -272,25 +267,25 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { * * @param sampleWellStr sampleSet decorate value * @param setWeight the sampleSet weight computed - * @param sampletWeight the sampleSet weight computed + * @param sampleWeight the sampleSet weight computed * @param useRfMinus10AndRfPlus10 flag to try to use rf-10 and rf+10 * @return the computed rf context for the given sample set */ - private RFContext computeRFContext(String sampleWellStr, SampleWellSetWeight setWeight, SampleWellSampleWeight sampletWeight, boolean useRfMinus10AndRfPlus10) { + private RFContext computeRFContext(String sampleWellStr, SampleWellSetWeight setWeight, SampleWellSampleWeight sampleWeight, boolean useRfMinus10AndRfPlus10) { float pondt = setWeight.getTotalWeight(); Float pondp = setWeight.getPlus10Weight(); Float pondm = setWeight.getMinus10Weight(); // compute rftot - float pdecht = sampletWeight.getTotalWeight(); + float pdecht = sampleWeight.getTotalWeight(); float rftot = pondt / pdecht; // compute rfMinus10 - Float pdechm = sampletWeight.getMinus10Weight(); + Float pdechm = sampleWeight.getMinus10Weight(); Float rfMinus10 = null; if (pdechm != null && pdechm > 0 && pondm != null) { rfMinus10 = pondm / pdechm; } // compute rfPlus10 - Float pdechp = sampletWeight.getPlus10Weight(); + Float pdechp = sampleWeight.getPlus10Weight(); Float rfPlus10 = null; if (pdechp != null && pdechp > 0 && pondp != null) { rfPlus10 = pondp / pdechp; @@ -310,7 +305,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { rfMinus10Status = RfUsageStatus.REJECTED_RF_NOT_DEFINED; } else { int rfMinus10MinNumber = getConfiguration().getRfMinus10MinNumber(); - Float sampletWeightMinus10Number = sampletWeight.getMinus10Number(); + Float sampletWeightMinus10Number = sampleWeight.getMinus10Number(); if (Math.abs(rfMinus10) < 0.001f) { addWarningMessage(l(locale, "t3.level1.extrapolateSampleWeightToSet.warning.rfMinus10.too.low", sampleWellStr, rfMinus10, rftot)); @@ -331,7 +326,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { rfPlus10Status = RfUsageStatus.REJECTED_RF_NOT_DEFINED; } else { int rfPlus10MinNumber = getConfiguration().getRfPlus10MinNumber(); - Float sampletWeightPlus10Number = sampletWeight.getPlus10Number(); + Float sampletWeightPlus10Number = sampleWeight.getPlus10Number(); if (Math.abs(rfPlus10) < 0.001f) { addWarningMessage(l(locale, "t3.level1.extrapolateSampleWeightToSet.warning.rfPlus10.too.low", sampleWellStr, rfPlus10, rftot)); @@ -387,25 +382,21 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { } /** - * Obtain for each species used in one of the given sample the first length - * class which represents a +10Kg weight. + * Obtain for each species used in one of the given sample the first length class which represents a +10Kg weight. * - * @param trip the trip where samples are done - * @param samples the sample to scan - * @param ocean ocean where trip happens + * @param species species to scan + * @param ocean where (used to get conversion) + * @param date when (used to get conversion) * @return the universe of length class limits computed indexed by species */ - private Map<Species, Integer> getThredHoldPlus10ForSpecies(Trip trip, Collection<Sample> samples, Ocean ocean) { - Set<Species> species = SpeciesTopiaDao.getAllSpeciesFromSampleSpecies(samples); + private Map<Species, Integer> getThresholdPlus10ForSpecies(Set<Species> species, Ocean ocean, Date date) { // for each of those species, found the length class which matches a weight > 10Kg Map<Species, Integer> result = new HashMap<>(); for (Species specie : species) { - LengthWeightConversion conversion = conversionHelper.getConversions(specie, ocean, 0, trip.getLandingDate()); + LengthWeightConversion conversion = conversionHelper.getConversions(specie, ocean, 0, date); if (conversion != null) { // only add conversion for species if found - WeightCategorySample fakeCategory = new WeightCategorySampleImpl(); - fakeCategory.setMax(10); - int lengthClass = conversionHelper.getSpecieHighestLengthClass(conversion, fakeCategory); + int lengthClass = conversionHelper.getSpecieHighestLengthClass(conversion, 10); result.put(specie, lengthClass); } } @@ -417,14 +408,6 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { return result; } -// private void setCategoryM10(WeightCategoryWellPlan categoryM10) { -// this.categoryM10 = categoryM10; -// } -// -// private void setCategoryP10(WeightCategoryWellPlan categoryP10) { -// this.categoryP10 = categoryP10; -// } - /** * This object contains the sample weights (-10,+10 and total) for sample well. * <ul> @@ -436,9 +419,9 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { */ protected static class SampleWellSampleWeight { - /** sum of fishes wieght for length class < +10Kg. */ + /** sum of fishes weight for length class < +10Kg. */ Float minus10Weight; - /** sum of fishes wieght for length class > +10Kg. */ + /** sum of fishes weight for length class > +10Kg. */ Float plus10Weight; /** Number of fishes for length class > -10Kg. */ private Float plus10Number; @@ -476,8 +459,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { } /** - * This object contains the set weights (-10,+10 and total) for - * sample well. + * This object contains the set weights (-10,+10 and total) for sample well. * <ul> * <li>{@link #getMinus10Weight()} is legacy {@code pondm}</li> * <li>{@link #getPlus10Weight()}} is legacy {@code pondp}</li> @@ -486,11 +468,11 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { */ protected static class SampleWellSetWeight { - /** sum of fishes wieght for length class < +10Kg. */ + /** sum of fishes weight for length class < +10Kg. */ Float totalWeight; - /** sum of fishes wieght for length class < +10Kg. */ + /** sum of fishes weight for length class < +10Kg. */ Float minus10Weight; - /** sum of fishes wieght for length class > +10Kg. */ + /** sum of fishes weight for length class > +10Kg. */ Float plus10Weight; protected static SampleWellSetWeight create(Float totalWeight, Float minus10Weight, Float plus10Weight) { ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratum.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratum.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratum.java @@ -8,12 +8,12 @@ * it under the terms of the GNU Affero General Public License as published by * the Free Software Foundation, either version 3 of the License, or * (at your option) any later version. - * + * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. - * + * * You should have received a copy of the GNU Affero General Public License * along with this program. If not, see <http://www.gnu.org/licenses/>. * #L% @@ -37,6 +37,8 @@ import fr.ird.t3.models.WeightCompositionModel; import fr.ird.t3.models.WeightCompositionModelHelper; import fr.ird.t3.services.DecoratorService; import fr.ird.t3.services.T3ServiceContext; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; import java.io.IOException; import java.util.Collection; @@ -55,6 +57,7 @@ import static org.nuiton.i18n.I18n.l; */ public class L2CatchStratum extends CatchStratum<Level2Configuration, Level2Action> { + private static final Log log = LogFactory.getLog(L2CatchStratum.class); /** * All species used by all weight categories found in all catches * for this stratum. @@ -100,13 +103,38 @@ public class L2CatchStratum extends CatchStratum<Level2Configuration, Level2Acti */ private WeightCompositionAggregateModel inputModelForSpeciesToFix; - L2CatchStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, Collection<Species> speciesToFix) { + private L2CatchStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, Collection<Species> speciesToFix) { super(stratumConfiguration, speciesToFix); weightCategoriesForSpecies = HashMultimap.create(); inputModelForAllSpecies = new WeightCompositionAggregateModel(); outputModelForAllSpecies = new WeightCompositionAggregateModel(); } + static L2CatchStratum newCatchStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, + List<WeightCategoryTreatment> weightCategories, + Collection<Species> species, + Level2Action action) throws Exception { + L2CatchStratum catchStratum = new L2CatchStratum(stratumConfiguration, species); + T3ServiceContext serviceContext = action.getServiceContext(); + catchStratum.init(serviceContext, weightCategories, action); + // get the total weight of the catch stratum + float catchStratumWeight = catchStratum.getTotalCatchWeightForSpeciesToFix(); + if (catchStratumWeight == 0) { + // no catch in this stratum, skip it + String message = l(serviceContext.getLocale(), "t3.level2.message.noCatch.in.stratum"); + log.info(message); + action.addInfoMessage(message); + // let's nullify the catch stratum (make it no more available) + catchStratum = null; + } else { + // log it + String message = catchStratum.logCatchStratum(action.getDecoratorService()); + log.info(message); + action.addInfoMessage(message); + } + return catchStratum; + } + @Override public void close() throws IOException { super.close(); @@ -130,11 +158,13 @@ public class L2CatchStratum extends CatchStratum<Level2Configuration, Level2Acti Integer nbZones = e.getValue(); if (activity.isCorrectedElementaryCatchNotEmpty()) { + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie for (CorrectedElementaryCatch aCatch : activity.getCorrectedElementaryCatch()) { weightCategoriesForSpecies.put(aCatch.getWeightCategoryTreatment(), aCatch.getSpecies()); } } + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie Multimap<WeightCategoryTreatment, CorrectedElementaryCatch> correctedElementaryCatchesByCategory = ActivityTopiaDao.groupByWeightCategoryTreatment(activity.getCorrectedElementaryCatch()); ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratumLoader.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratumLoader.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratumLoader.java @@ -25,8 +25,10 @@ import fr.ird.t3.actions.stratum.CatchStratumLoader; import fr.ird.t3.actions.stratum.StratumConfiguration; import fr.ird.t3.entities.data.Activity; import fr.ird.t3.entities.data.ActivityTopiaDao; +import fr.ird.t3.entities.data.CorrectedElementaryCatch; import fr.ird.t3.entities.data.Trip; import fr.ird.t3.entities.reference.Vessel; +import fr.ird.t3.entities.reference.WeightCategoryTreatment; import fr.ird.t3.services.ioc.InjectDAO; import java.util.HashMap; @@ -71,6 +73,7 @@ public class L2CatchStratumLoader extends CatchStratumLoader<Level2Configuration Map<Activity, Integer> result = new HashMap<>(); if (activityIds != null && activityIds.size() > 0) { Set<Vessel> possibleVessels = configuration.getPossibleCatchVessels(); + WeightCategoryTreatment weightCategoryTreatment = configuration.getWeightCategoryTreatment(); for (Map.Entry<String, Integer> e : activityIds.entrySet()) { String activityId = e.getKey(); // get activity @@ -84,6 +87,14 @@ public class L2CatchStratumLoader extends CatchStratumLoader<Level2Configuration // recheck activity have some catches. Preconditions.checkState(activity.isCorrectedElementaryCatchNotEmpty(), String.format("Can not accept an activity (%s) with no catch", activity.getTopiaId())); + if (weightCategoryTreatment != null) { + // weight category is in stratum, check there is catch using this category + boolean foundWeightCategory = activity.getCorrectedElementaryCatch().stream().map(CorrectedElementaryCatch::getWeightCategoryTreatment).anyMatch(c -> c.equals(weightCategoryTreatment)); + if (!foundWeightCategory) { + // reject - the stratum weight category not found for this activity + continue; + } + } result.put(activity, e.getValue()); } } ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratum.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratum.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratum.java @@ -26,11 +26,12 @@ import fr.ird.t3.entities.data.Activity; import fr.ird.t3.entities.data.ActivityTopiaDao; import fr.ird.t3.entities.data.SetSpeciesFrequency; import fr.ird.t3.entities.reference.Species; +import fr.ird.t3.entities.reference.WeightCategory; import fr.ird.t3.entities.reference.WeightCategorySample; +import fr.ird.t3.entities.reference.WeightCategoryTreatment; import fr.ird.t3.models.WeightCompositionAggregateModel; import fr.ird.t3.models.WeightCompositionModelHelper; import fr.ird.t3.services.T3ServiceContext; -import org.apache.commons.collections.CollectionUtils; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.nuiton.topia.persistence.TopiaException; @@ -38,6 +39,7 @@ import org.nuiton.topia.persistence.TopiaException; import java.io.IOException; import java.util.Collection; import java.util.HashMap; +import java.util.List; import java.util.Locale; import java.util.Map; import java.util.Set; @@ -81,12 +83,22 @@ public class L2SampleStratum extends SampleStratum<Level2Configuration, Level2Ac */ private float sampleStratumTotalWeight; - L2SampleStratum(StratumConfiguration<Level2Configuration> configuration, Collection<Species> speciesToFix, float catchStratumTotalWeight) { + private L2SampleStratum(StratumConfiguration<Level2Configuration> configuration, Collection<Species> speciesToFix, float catchStratumTotalWeight) { super(configuration, speciesToFix); this.catchStratumTotalWeight = catchStratumTotalWeight; modelsForAllSpecies = new WeightCompositionAggregateModel(); } + static L2SampleStratum newSampleStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, + List<WeightCategoryTreatment> weightCategories, + float totalCatchWeight, + Collection<Species> species, + Level2Action action) throws Exception { + L2SampleStratum sampleStratum = new L2SampleStratum(stratumConfiguration, species, totalCatchWeight); + sampleStratum.init(action.getServiceContext(), weightCategories, action); + return sampleStratum; + } + @Override public void close() throws IOException { super.close(); @@ -97,18 +109,14 @@ public class L2SampleStratum extends SampleStratum<Level2Configuration, Level2Ac @Override protected L2SampleStratumLoader newLoader() { int oceanCode = getConfiguration().getZone().getOcean().getCode(); - L2SampleStratumLoader result; switch (oceanCode) { case 1: - result = new L2SampleStratumLoaderAtlantic(this); - break; + return new L2SampleStratumLoaderAtlantic(this); case 2: - result = new L2SampleStratumLoaderIndian(this); - break; + return new L2SampleStratumLoaderIndian(this); default: throw new IllegalStateException("Not implemented for ocean with code " + oceanCode); } - return result; } public WeightCompositionAggregateModel getModelsForSpeciesToFix() { @@ -137,7 +145,7 @@ public class L2SampleStratum extends SampleStratum<Level2Configuration, Level2Ac ActivityTopiaDao.fillWeightsFromSetSpeciesCatWeight(activity, weights, null); // obtain the set species frequencies for the current activity Collection<SetSpeciesFrequency> setSpeciesFrequencies = activity.getSetSpeciesFrequency(); - if (CollectionUtils.isNotEmpty(setSpeciesFrequencies)) { + if (activity.isSetSpeciesFrequencyNotEmpty()) { // compute sample count for this activity int newCount = computeSampleCount(setSpeciesFrequencies, species); // merge it with final total count @@ -145,24 +153,25 @@ public class L2SampleStratum extends SampleStratum<Level2Configuration, Level2Ac } } // add all weights to model - for (WeightCategorySample weightCategoryTreatment : weights.keySet()) { - Map<Species, Float> speciesFloatMap = weights.get(weightCategoryTreatment); - modelsForAllSpecies.addModel(weightCategoryTreatment, speciesFloatMap); + for (Map.Entry<WeightCategorySample, Map<Species, Float>> e : weights.entrySet()) { + Map<Species, Float> speciesFloatMap = e.getValue(); + WeightCategory weightCategorySample = e.getKey(); + modelsForAllSpecies.addModel(weightCategorySample, speciesFloatMap); } // recompute the weight model for species to fix modelsForSpeciesToFix = modelsForAllSpecies.extractForSpecies(species); // recompute the total sample weight (for species to fix) sampleStratumTotalWeight = modelsForSpeciesToFix.getTotalModel().getTotalWeight(); - addMergedActivitesCount(activities.size()); + addMergedActivitiesCount(activities.size()); log.info(String.format("sampleStratumTotalCount = %d / sampleStratumTotalWeight = %f", getSampleStratumTotalCount(), getSampleStratumTotalWeight())); } - private int computeSampleCount(Collection<SetSpeciesFrequency> newDatas, Collection<Species> speciesToFix) { + private int computeSampleCount(Collection<SetSpeciesFrequency> setSpeciesFrequencies, Collection<Species> speciesToFix) { int newCount = 0; - for (SetSpeciesFrequency newData : newDatas) { - Species species = newData.getSpecies(); + for (SetSpeciesFrequency setSpeciesFrequency : setSpeciesFrequencies) { + Species species = setSpeciesFrequency.getSpecies(); if (speciesToFix.contains(species)) { - newCount += newData.getNumber(); + newCount += setSpeciesFrequency.getNumber(); } } return newCount; ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2Action.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2Action.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2Action.java @@ -51,6 +51,7 @@ import fr.ird.t3.models.WeightCompositionAggregateModel; import fr.ird.t3.models.WeightCompositionModel; import fr.ird.t3.models.WeightCompositionModelHelper; import fr.ird.t3.services.DecoratorService; +import fr.ird.t3.services.T3ServiceContext; import fr.ird.t3.services.ZoneStratumService; import fr.ird.t3.services.ioc.InjectDAO; import fr.ird.t3.services.ioc.InjectEntitiesById; @@ -113,7 +114,7 @@ public class Level2Action extends T3Action<Level2Configuration> { * * @since 1.3 */ - private Collection<L2StratumResult> stratumsResult; + private Collection<L2StratumResult> stratumResultSet; @InjectDAO(entityType = Activity.class) private ActivityTopiaDao activityDAO; @InjectDAO(entityType = WeightCategoryTreatment.class) @@ -179,7 +180,7 @@ public class Level2Action extends T3Action<Level2Configuration> { this.possibleCatchVessels = vesselDAO.getPossibleCatchVessels(catchFleets); // get possible vessels for sample stratum this.possibleSampleVessels = vesselDAO.getPossibleSampleVessels(sampleFleets, sampleFlags); - this.stratumsResult = new LinkedHashSet<>(); + this.stratumResultSet = new LinkedHashSet<>(); this.inputCatchModelForAllSpecies = new WeightCompositionAggregateModel(); this.outputCatchModelForAllSpecies = new WeightCompositionAggregateModel(); } @@ -278,7 +279,7 @@ public class Level2Action extends T3Action<Level2Configuration> { activityCache); try { L2StratumResult result = doExecuteStratum(stratumConfiguration, weightCategories, stratumIndex, usedActivityIds); - stratumsResult.add(result); + stratumResultSet.add(result); } finally { flushTransaction("After stratum " + stratumIndex); } @@ -319,7 +320,7 @@ public class Level2Action extends T3Action<Level2Configuration> { addInfoMessage(message); addInfoMessage("=============================================================================================="); // compute the catch stratum - try (L2CatchStratum catchStratum = newCatchStratum(stratumConfiguration, weightCategories)) { + try (L2CatchStratum catchStratum = L2CatchStratum.newCatchStratum(stratumConfiguration, weightCategories, species, this)) { incrementsProgression(); if (catchStratum == null) { // no catch in this stratum @@ -328,7 +329,7 @@ public class Level2Action extends T3Action<Level2Configuration> { incrementsProgression(); } else { // compute sample stratum - try (L2SampleStratum sampleStratum = newSampleStratum(stratumConfiguration, weightCategories, catchStratum.getTotalCatchWeightForSpeciesToFix())) { + try (L2SampleStratum sampleStratum = L2SampleStratum.newSampleStratum(stratumConfiguration, weightCategories, catchStratum.getTotalCatchWeightForSpeciesToFix(), species, this)) { incrementsProgression(); // get the substitution level for the sample stratum Integer level = sampleStratum.getSubstitutionLevel(); @@ -349,7 +350,7 @@ public class Level2Action extends T3Action<Level2Configuration> { // int nbZones = e.getValue(); // is activity was already treated ? boolean newActivity = usedActivityIds.add(activity.getTopiaId()); - doExecuteActivityInCatchStratum(catchStratum, + doExecuteActivityInCatchStratumWithoutSampleStratum(catchStratum, activity, activityIndex++, nbActivities, @@ -409,34 +410,73 @@ public class Level2Action extends T3Action<Level2Configuration> { return oceanContext.values().stream().flatMap(o -> o.getZones(schoolType).stream()).distinct().collect(Collectors.toList()); } - private L2CatchStratum newCatchStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, - List<WeightCategoryTreatment> weightCategories) throws Exception { - L2CatchStratum catchStratum = new L2CatchStratum(stratumConfiguration, species); - catchStratum.init(serviceContext, weightCategories, this); - // get the total weight of the catch stratum - float catchStratumWeight = catchStratum.getTotalCatchWeightForSpeciesToFix(); - if (catchStratumWeight == 0) { - // no catch in this stratum, skip it - String message = l(locale, "t3.level2.message.noCatch.in.stratum"); - log.info(message); - addInfoMessage(message); - // let's nullify the catch stratum (make it no more available) - catchStratum = null; - } else { - // log it - String message = catchStratum.logCatchStratum(getDecoratorService()); - log.info(message); - addInfoMessage(message); + private void doExecuteActivityInCatchStratumWithoutSampleStratum(L2CatchStratum catchStratum, + Activity activity, + int activityIndex, + int nbActivities, + boolean deleteOldData) { + + String activityStr = String.format("%s (%s)", decorate(activity), decorate(activity.getTrip(), DecoratorService.WITH_ID)); + String message = l(locale, "t3.level2.message.start.activity", activityIndex, nbActivities, activityStr, 1); + log.info(message); + addInfoMessage(message); + if (deleteOldData) { + // delete old data for this activity + log.info(String.format("Delete previous level2 data of %s", activityStr)); + activity.deleteComputedDataLevel2(); + } + // build a model for getting total of each weight category + try (WeightCompositionAggregateModel catchWeightModelForAllSpecies = new WeightCompositionAggregateModel()) { + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie + ActivityTopiaDao.fillWeightsFromCatchesWeight(activity, catchWeightModelForAllSpecies, 1); + try (WeightCompositionAggregateModel catchWeightModelForSpeciesToFix = catchWeightModelForAllSpecies.extractForSpecies(species)) { + // log catches weight + message = logCatchWeight(getDecoratorService(), catchWeightModelForAllSpecies, catchWeightModelForSpeciesToFix); + log.info(message); + addInfoMessage(message); + } + try (WeightCompositionAggregateModel model = new WeightCompositionAggregateModel()) { + ActivityTopiaDao.fillWeightsFromSetSpeciesCatWeight(activity, null, model); + try (WeightCompositionAggregateModel compositionModel = model.extractForSpecies(species)) { + String activityResume = logActivityCatchStratum(model, compositionModel, getDecoratorService()); + message = l(locale, "t3.level2.message.activity.with.sample.useOwn.composition", activityResume); + log.info(message); + addInfoMessage(message); + } + } + // apply composition model to activity catches + applySampleSpecificCompositionWithoutSampleStratum(catchStratum, activity); } - return catchStratum; } - private L2SampleStratum newSampleStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, - List<WeightCategoryTreatment> weightCategories, - float totalCatchWeight) throws Exception { - L2SampleStratum sampleStratum = new L2SampleStratum(stratumConfiguration, species, totalCatchWeight); - sampleStratum.init(serviceContext, weightCategories, this); - return sampleStratum; + private void applySampleSpecificCompositionWithoutSampleStratum(L2CatchStratum catchStratum, Activity activity) { + try (WeightCompositionAggregateModel correctedCatchWeightModel = new WeightCompositionAggregateModel()) { + // Get all corrected catch group by weight category + Multimap<WeightCategoryTreatment, CorrectedElementaryCatch> correctedElementaryCatches = + ActivityTopiaDao.groupByWeightCategoryTreatment(activity.getCorrectedElementaryCatch()); + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie + for (WeightCategoryTreatment weightCategory : correctedElementaryCatches.keySet()) { + Map<Species, Float> weights = new HashMap<>(); + // Apply on all existing corrected catch weight + for (CorrectedElementaryCatch aCatch : correctedElementaryCatches.get(weightCategory)) { + Species speciesToUse = aCatch.getSpecies(); + // the corrected catch weight to add to the row + float correctedCatchWeight = aCatch.getCatchWeight(); + // hold added value (for logs) + weights.put(speciesToUse, correctedCatchWeight); + aCatch.setCorrectedCatchWeight(correctedCatchWeight); + aCatch.setCorrectedFlag(false); + } + // add corrected weights used for this category + correctedCatchWeightModel.addModel(weightCategory, weights); + } + // log corrected catches weight + String message = logCorrectedCatchWeight(correctedCatchWeightModel, getDecoratorService()); + log.info(message); + addInfoMessage(message); + // add corrected catch weight done for this activity in stratum output model + catchStratum.addActivityOutputModel(correctedCatchWeightModel); + } } private void doExecuteActivityInCatchStratum(L2CatchStratum catchStratum, @@ -457,14 +497,14 @@ public class Level2Action extends T3Action<Level2Configuration> { } // build a model for getting total of each weight category WeightCompositionAggregateModel catchWeightModelForAllSpecies = new WeightCompositionAggregateModel(); + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie ActivityTopiaDao.fillWeightsFromCatchesWeight(activity, catchWeightModelForAllSpecies, nbZones); WeightCompositionAggregateModel catchWeightModelForSpeciesToFix = catchWeightModelForAllSpecies.extractForSpecies(species); // log catches weight message = logCatchWeight(getDecoratorService(), catchWeightModelForAllSpecies, catchWeightModelForSpeciesToFix); log.info(message); addInfoMessage(message); - // obtain the sample composition model to use for the activity (can come from catch if it has some samples, - // otherwise use the catch stratum one) + // obtain the sample composition model to use for the activity (can come from catch if it has some samples, otherwise use the catch stratum one) boolean activityWithSample = catchStratum.isActivityWithSample(activity); boolean useAllSamplesOfStratum = getConfiguration().isUseAllSamplesOfStratum(); WeightCompositionAggregateModel compositionModel; @@ -498,161 +538,98 @@ public class Level2Action extends T3Action<Level2Configuration> { } } - private void doExecuteActivityInCatchStratum(L2CatchStratum catchStratum, - Activity activity, - int activityIndex, - int nbActivities, - boolean deleteOldData) { - - String activityStr = String.format("%s (%s)", decorate(activity), decorate(activity.getTrip(), DecoratorService.WITH_ID)); - String message = l(locale, "t3.level2.message.start.activity", activityIndex, nbActivities, activityStr, 1); - log.info(message); - addInfoMessage(message); - if (deleteOldData) { - // delete old data for this activity - log.info(String.format("Delete previous level2 data of %s", activityStr)); - activity.deleteComputedDataLevel2(); - } - // build a model for getting total of each weight category - WeightCompositionAggregateModel catchWeightModelForAllSpecies = new WeightCompositionAggregateModel(); - ActivityTopiaDao.fillWeightsFromCatchesWeight(activity, catchWeightModelForAllSpecies, 1); - WeightCompositionAggregateModel catchWeightModelForSpeciesToFix = catchWeightModelForAllSpecies.extractForSpecies(species); - // log catches weight - message = logCatchWeight(getDecoratorService(), catchWeightModelForAllSpecies, catchWeightModelForSpeciesToFix); - log.info(message); - addInfoMessage(message); - WeightCompositionAggregateModel model = new WeightCompositionAggregateModel(); - ActivityTopiaDao.fillWeightsFromSetSpeciesCatWeight(activity, null, model); - WeightCompositionAggregateModel compositionModel = model.extractForSpecies(species); - String activityResume = logActivityCatchStratum(model, compositionModel, getDecoratorService()); - message = l(locale, "t3.level2.message.activity.with.sample.useOwn.composition", activityResume); - log.info(message); - addInfoMessage(message); - // apply composition model to activity catches - applySampleSpecificComposition(catchStratum, activity); - catchWeightModelForAllSpecies.close(); - catchWeightModelForSpeciesToFix.close(); - compositionModel.close(); - } - - private void applySampleSpecificComposition(L2CatchStratum catchStratum, Activity activity) { - WeightCompositionAggregateModel correctedCatchWeightModel = new WeightCompositionAggregateModel(); - // Get all corrected catch group by weight category - Multimap<WeightCategoryTreatment, CorrectedElementaryCatch> correctedElementaryCatches = - ActivityTopiaDao.groupByWeightCategoryTreatment(activity.getCorrectedElementaryCatch()); - for (WeightCategoryTreatment weightCategory : correctedElementaryCatches.keySet()) { - Map<Species, Float> weights = new HashMap<>(); - // Apply on all existing corrected catch weight - for (CorrectedElementaryCatch aCatch : correctedElementaryCatches.get(weightCategory)) { - Species speciesToUse = aCatch.getSpecies(); - // the corrected catch weight to add to the row - float correctedCatchWeight = aCatch.getCatchWeight(); - // hold added value (for logs) - weights.put(speciesToUse, correctedCatchWeight); - aCatch.setCorrectedCatchWeight(correctedCatchWeight); - aCatch.setCorrectedFlag(false); - } - // add corrected weights used for this category - correctedCatchWeightModel.addModel(weightCategory, weights); - } - // log corrected catches weight - String message = logCorrectedCatchWeight(correctedCatchWeightModel, getDecoratorService()); - log.info(message); - addInfoMessage(message); - // add corrected catch weight done for this activity in stratum output model - catchStratum.addActivityOutputModel(correctedCatchWeightModel); - } - private void applySampleSpecificComposition(L2CatchStratum catchStratum, Activity activity, int nbZones, WeightCompositionAggregateModel sampleCompositionModel, WeightCompositionAggregateModel correctedCatchesForSpeciesToFix) { - WeightCompositionAggregateModel correctedCatchWeightModel = new WeightCompositionAggregateModel(); - // Get all corrected catch group by weight category - Multimap<WeightCategoryTreatment, CorrectedElementaryCatch> correctedElementaryCatches = - ActivityTopiaDao.groupByWeightCategoryTreatment(activity.getCorrectedElementaryCatch()); - for (WeightCategoryTreatment weightCategory : correctedElementaryCatches.keySet()) { - Map<Species, Float> weights = new HashMap<>(); - // get composition model to use - WeightCompositionModel model = sampleCompositionModel.getModel(weightCategory); - // set of species still to fix - Set<Species> speciesToFix; - // set of species still to fix - Set<Species> speciesStillToFix; - // get total weight for species to fix from correctedCatches - float totalWeight; - if (model == null) { - // this means there is no specific composition for this weight category to apply for species to fix - // says there is no species to fix (so all catches will be copied to correctedCatchWeight (with no fixedFlag) - speciesToFix = speciesStillToFix = Collections.emptySet(); - totalWeight = 0f; - } else { - // found a specific composition to apply for this weight category - speciesToFix = model.getSpecies(); - speciesStillToFix = new HashSet<>(speciesToFix); - totalWeight = correctedCatchesForSpeciesToFix.getModel(weightCategory).getTotalWeight(); - } - // divide the total weight to used by the number of zones of this activity - totalWeight = totalWeight / nbZones; - // Apply on all existing corrected catch weight - for (CorrectedElementaryCatch aCatch : correctedElementaryCatches.get(weightCategory)) { - Species speciesToUse = Objects.requireNonNull(aCatch.getSpecies()); - // flag (was fixed or not) - boolean toFix; - // the corrected catch weight to add to the row - float correctedCatchWeight; - if (model != null && speciesToFix.contains(speciesToUse)) { - // species to fix - toFix = true; - float weightRate = model.getWeightRate(speciesToUse); - // new corrected catch weight - correctedCatchWeight = totalWeight * weightRate; - // species no more to treat - speciesStillToFix.remove(speciesToUse); + try (WeightCompositionAggregateModel correctedCatchWeightModel = new WeightCompositionAggregateModel()) { + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie + // Get all corrected catch group by weight category + Multimap<WeightCategoryTreatment, CorrectedElementaryCatch> correctedElementaryCatches = + ActivityTopiaDao.groupByWeightCategoryTreatment(activity.getCorrectedElementaryCatch()); + for (WeightCategoryTreatment weightCategory : correctedElementaryCatches.keySet()) { + Map<Species, Float> weights = new HashMap<>(); + // get composition model to use + WeightCompositionModel model = sampleCompositionModel.getModel(weightCategory); + // set of species still to fix + Set<Species> speciesToFix; + // set of species still to fix + Set<Species> speciesStillToFix; + // get total weight for species to fix from correctedCatches + float totalWeight; + if (model == null) { + // this means there is no specific composition for this weight category to apply for species to fix + // says there is no species to fix (so all catches will be copied to correctedCatchWeight (with no fixedFlag) + speciesToFix = speciesStillToFix = Collections.emptySet(); + totalWeight = 0f; } else { - // nothing to fix, just propagate old value - toFix = false; - correctedCatchWeight = aCatch.getCatchWeight(); + // found a specific composition to apply for this weight category + speciesToFix = model.getSpecies(); + speciesStillToFix = new HashSet<>(speciesToFix); + totalWeight = correctedCatchesForSpeciesToFix.getModel(weightCategory).getTotalWeight(); + } + // divide the total weight to used by the number of zones of this activity + totalWeight = totalWeight / nbZones; + // Apply on all existing corrected catch weight + for (CorrectedElementaryCatch aCatch : correctedElementaryCatches.get(weightCategory)) { + Species speciesToUse = Objects.requireNonNull(aCatch.getSpecies()); + // flag (was fixed or not) + boolean toFix; + // the corrected catch weight to add to the row + float correctedCatchWeight; + if (model != null && speciesToFix.contains(speciesToUse)) { + // species to fix + toFix = true; + float weightRate = model.getWeightRate(speciesToUse); + // new corrected catch weight + correctedCatchWeight = totalWeight * weightRate; + // species no more to treat + speciesStillToFix.remove(speciesToUse); + } else { + // nothing to fix, just propagate old value + toFix = false; + correctedCatchWeight = aCatch.getCatchWeight(); + } + // hold added value (for logs) + weights.put(speciesToUse, correctedCatchWeight); + // Add to old value (if any) + Float oldCorrectedCatchWeight = aCatch.getCorrectedCatchWeight(); + if (oldCorrectedCatchWeight == null) { + oldCorrectedCatchWeight = 0f; + } + aCatch.setCorrectedCatchWeight(oldCorrectedCatchWeight + correctedCatchWeight); + aCatch.setCorrectedFlag(toFix); } - // hold added value (for logs) - weights.put(speciesToUse, correctedCatchWeight); - // Add to old value (if any) - Float oldCorrectedCatchWeight = aCatch.getCorrectedCatchWeight(); - if (oldCorrectedCatchWeight == null) { - oldCorrectedCatchWeight = 0f; + // Creates new corrected catch weight records for all species still to fix + // Means they are in samples but not in catches + for (Species speciesToUse : speciesStillToFix) { + // weight rate of the species + float weightRate = model == null ? 1f : model.getWeightRate(speciesToUse); + // corrected catch weight + float correctedCatchWeight = totalWeight * weightRate; + // create new record + CorrectedElementaryCatch aCatch = correctedElementaryCatchDAO.create(); + aCatch.setSpecies(speciesToUse); + aCatch.setWeightCategoryTreatment(weightCategory); + aCatch.setCatchWeight(0f); + aCatch.setCorrectedFlag(true); + aCatch.setCorrectedCatchWeight(correctedCatchWeight); + // hold added value (for logs) + weights.put(speciesToUse, correctedCatchWeight); + // add it to activity + activity.addCorrectedElementaryCatch(aCatch); } - aCatch.setCorrectedCatchWeight(oldCorrectedCatchWeight + correctedCatchWeight); - aCatch.setCorrectedFlag(toFix); - } - // Creates new corrected catch weight records for all species still to fix - // Means they are in samples but not in catches - for (Species speciesToUse : speciesStillToFix) { - // weight rate of the species - float weightRate = model == null ? 1f : model.getWeightRate(speciesToUse); - // corrected catch weight - float correctedCatchWeight = totalWeight * weightRate; - // create new record - CorrectedElementaryCatch aCatch = correctedElementaryCatchDAO.create(); - aCatch.setSpecies(speciesToUse); - aCatch.setWeightCategoryTreatment(weightCategory); - aCatch.setCatchWeight(0f); - aCatch.setCorrectedFlag(true); - aCatch.setCorrectedCatchWeight(correctedCatchWeight); - // hold added value (for logs) - weights.put(speciesToUse, correctedCatchWeight); - // add it to activity - activity.addCorrectedElementaryCatch(aCatch); + // add corrected weights used for this category + correctedCatchWeightModel.addModel(weightCategory, weights); } - // add corrected weights used for this category - correctedCatchWeightModel.addModel(weightCategory, weights); + // log corrected catches weight + String message = logCorrectedCatchWeight(correctedCatchWeightModel, getDecoratorService()); + log.info(message); + addInfoMessage(message); + // add corrected catch weight done for this activity in stratum output model + catchStratum.addActivityOutputModel(correctedCatchWeightModel); } - // log corrected catches weight - String message = logCorrectedCatchWeight(correctedCatchWeightModel, getDecoratorService()); - log.info(message); - addInfoMessage(message); - // add corrected catch weight done for this activity in stratum output model - catchStratum.addActivityOutputModel(correctedCatchWeightModel); } // ------------------------------------------------------------------------- @@ -681,13 +658,13 @@ public class Level2Action extends T3Action<Level2Configuration> { @SuppressWarnings("unused") public int getNbStrataFixed() { - return stratumsResult.size(); + return stratumResultSet.size(); } @SuppressWarnings({"unused", "WeakerAccess"}) public Integer[] getAllSubstitutionLevels() { Set<Integer> levels = new HashSet<>(); - for (L2StratumResult stratumResult : stratumsResult) { + for (L2StratumResult stratumResult : stratumResultSet) { levels.add(stratumResult.getSubstitutionLevel()); } List<Integer> result = new ArrayList<>(levels); @@ -703,7 +680,7 @@ public class Level2Action extends T3Action<Level2Configuration> { @SuppressWarnings("unused") public Collection<L2StratumResult> getStratumResult(int level) { Set<L2StratumResult> singleResult = new LinkedHashSet<>(); - for (L2StratumResult stratumResult : stratumsResult) { + for (L2StratumResult stratumResult : stratumResultSet) { if (level == stratumResult.getSubstitutionLevel()) { singleResult.add(stratumResult); } @@ -717,7 +694,7 @@ public class Level2Action extends T3Action<Level2Configuration> { Integer[] levels = getAllSubstitutionLevels(); for (Integer level : levels) { Set<L2StratumResult> singleResult = new LinkedHashSet<>(); - for (L2StratumResult stratumResult : stratumsResult) { + for (L2StratumResult stratumResult : stratumResultSet) { if (level == stratumResult.getSubstitutionLevel()) { singleResult.add(stratumResult); } @@ -752,7 +729,7 @@ public class Level2Action extends T3Action<Level2Configuration> { @SuppressWarnings("unused") public int getMaximumSizeForStratum() { int result = 0; - for (L2StratumResult stratumResult : stratumsResult) { + for (L2StratumResult stratumResult : stratumResultSet) { result = Math.max(result, stratumResult.getLibelle().length() + 1); } return result; @@ -796,8 +773,8 @@ public class Level2Action extends T3Action<Level2Configuration> { modelForSpeciestoFix); } - public void setStratumsResult(Set<L2StratumResult> stratumsResult) { - this.stratumsResult = stratumsResult; + public void setStratumResultSet(Set<L2StratumResult> stratumResultSet) { + this.stratumResultSet = stratumResultSet; } public void setInputCatchModelForAllSpecies(WeightCompositionAggregateModel inputCatchModelForAllSpecies) { @@ -807,4 +784,8 @@ public class Level2Action extends T3Action<Level2Configuration> { public void setOutputCatchModelForAllSpecies(WeightCompositionAggregateModel outputCatchModelForAllSpecies) { this.outputCatchModelForAllSpecies = outputCatchModelForAllSpecies; } + + T3ServiceContext getServiceContext() { + return serviceContext; + } } ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3SampleStratum.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3SampleStratum.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3SampleStratum.java @@ -181,7 +181,7 @@ public class L3SampleStratum extends SampleStratum<Level3Configuration, Level3Ac speciesCount.put(species, oldCount + newCount); } } - addMergedActivitesCount(activities.size()); + addMergedActivitiesCount(activities.size()); if (log.isInfoEnabled()) { log.info(String.format("sampleStratumTotalWeight = %s", getSampleStratumTotalWeight())); ===================================== t3-actions/src/test/java/fr/ird/t3/actions/ActionResumeTest.java ===================================== --- a/t3-actions/src/test/java/fr/ird/t3/actions/ActionResumeTest.java +++ b/t3-actions/src/test/java/fr/ird/t3/actions/ActionResumeTest.java @@ -746,7 +746,7 @@ public class ActionResumeTest<C extends T3ActionConfiguration, A extends T3Actio stratumResult.setNbActivities(5); stratumsResult.add(stratumResult); - action.setStratumsResult(stratumsResult); + action.setStratumResultSet(stratumsResult); action.setInputCatchModelForAllSpecies(new WeightCompositionAggregateModel()); action.setOutputCatchModelForAllSpecies(new WeightCompositionAggregateModel()); ===================================== t3-domain/src/main/java/fr/ird/t3/actions/stratum/SampleStratum.java ===================================== --- a/t3-domain/src/main/java/fr/ird/t3/actions/stratum/SampleStratum.java +++ b/t3-domain/src/main/java/fr/ird/t3/actions/stratum/SampleStratum.java @@ -27,7 +27,6 @@ import fr.ird.t3.entities.reference.WeightCategoryTreatment; import fr.ird.t3.services.IOCService; import fr.ird.t3.services.T3ServiceContext; import fr.ird.t3.services.ioc.InjectDAO; -import org.nuiton.topia.persistence.TopiaException; import java.util.Collection; import java.util.List; @@ -60,10 +59,13 @@ public abstract class SampleStratum<C extends LevelConfigurationWithStratum, A e */ private Integer substitutionLevel; + protected SampleStratum(StratumConfiguration<C> configuration, Collection<Species> speciesToFix) { + super(configuration, speciesToFix); + } + protected abstract SampleStratumLoader<C, A, S> newLoader(); - protected abstract String logSampleStratumLevel(int substitutionLevel, - A messager); + protected abstract String logSampleStratumLevel(int substitutionLevel, A messager); /** * Merge the given {@code activities} sample data in the stratum result. @@ -81,24 +83,15 @@ public abstract class SampleStratum<C extends LevelConfigurationWithStratum, A e * * @param serviceContext service context * @param activities the activities to merge - * @throws TopiaException if any database problem while loading data */ - protected abstract void mergeNewActivities(T3ServiceContext serviceContext, - Set<Activity> activities) throws TopiaException; - - protected SampleStratum(StratumConfiguration<C> configuration, - Collection<Species> speciesToFix) { - super(configuration, speciesToFix); - } + protected abstract void mergeNewActivities(T3ServiceContext serviceContext, Set<Activity> activities); - protected final void addMergedActivitesCount(int nb) { + protected final void addMergedActivitiesCount(int nb) { nbMergedActivities += nb; } @Override - public void init(T3ServiceContext serviceContext, - List<WeightCategoryTreatment> weightCategories, - A messager) throws Exception { + public void init(T3ServiceContext serviceContext, List<WeightCategoryTreatment> weightCategories, A messager) throws Exception { SampleStratumLoader<C, A, S> stratumLoader = newLoader(); ===================================== t3-domain/src/main/java/fr/ird/t3/entities/reference/LengthWeightConversionHelper.java ===================================== --- a/t3-domain/src/main/java/fr/ird/t3/entities/reference/LengthWeightConversionHelper.java +++ b/t3-domain/src/main/java/fr/ird/t3/entities/reference/LengthWeightConversionHelper.java @@ -40,28 +40,25 @@ import java.util.TreeMap; */ public class LengthWeightConversionHelper { - /** Logger. */ - private static final Log log = - LogFactory.getLog(LengthWeightConversionHelper.class); + private static final Log log = LogFactory.getLog(LengthWeightConversionHelper.class); - protected final LengthWeightConversionTopiaDao dao; + private final LengthWeightConversionTopiaDao dao; private final Map<String, LengthWeightConversion> lengthWeightConversions; private final Map<String, Integer> lengthClassMap; - private final ConversionContext conversionContext; LengthWeightConversionHelper(LengthWeightConversionTopiaDao dao, Ocean ocean, int sex, Date date) { this.dao = dao; this.conversionContext = new ConversionContext(ocean, sex, date); - lengthWeightConversions = new TreeMap<>(); - lengthClassMap = new TreeMap<>(); + this.lengthWeightConversions = new TreeMap<>(); + this.lengthClassMap = new TreeMap<>(); } LengthWeightConversionHelper(LengthWeightConversionTopiaDao dao) { this.dao = dao; this.conversionContext = null; - lengthWeightConversions = new TreeMap<>(); - lengthClassMap = new TreeMap<>(); + this.lengthWeightConversions = new TreeMap<>(); + this.lengthClassMap = new TreeMap<>(); } private <W extends WeightCategory> W getNextCategory(LengthWeightConversion conversion, int lengthClass, Iterator<W> itr) { @@ -77,7 +74,7 @@ public class LengthWeightConversionHelper { } else { // get max lengthClass - int nextLengthClass = getSpecieHighestLengthClass(conversion, result); + int nextLengthClass = getSpecieHighestLengthClass(conversion, result.getMax()); if (nextLengthClass < lengthClass) { @@ -89,10 +86,9 @@ public class LengthWeightConversionHelper { return result; } - public <W extends WeightCategory> Map<Integer, W> getWeightCategoriesDistribution( - LengthWeightConversion conversion, - List<W> weightCategories, - List<Integer> lengthClasses) { + public <W extends WeightCategory> Map<Integer, W> getWeightCategoriesDistribution(LengthWeightConversion conversion, + List<W> weightCategories, + List<Integer> lengthClasses) { Map<Integer, W> result = new TreeMap<>(); W currentWeightCategory = null; int currentMaxLengthClass = -1; @@ -105,7 +101,7 @@ public class LengthWeightConversionHelper { if (currentWeightCategory == null) { // get the correct weight category currentWeightCategory = getNextCategory(conversion, lengthClass, itr); - currentMaxLengthClass = getSpecieHighestLengthClass(conversion, currentWeightCategory); + currentMaxLengthClass = getSpecieHighestLengthClass(conversion, currentWeightCategory.getMax()); } result.put(lengthClass, currentWeightCategory); } @@ -113,7 +109,7 @@ public class LengthWeightConversionHelper { } /** - * Get the convertor for the given parameters from the cache. + * Get the converter for the given parameters from the cache. * <p/> * If not found, then load it from db. * @@ -122,13 +118,10 @@ public class LengthWeightConversionHelper { * @param sex sex to use * @param date min date to use * @return convertor found - * @throws TopiaException if any db while querying db */ public LengthWeightConversion getConversions(Species species, Ocean ocean, int sex, Date date) { - LengthWeightConversion result; - String key = species.getCode() + "-" + ocean.getCode() + - "-" + sex + "-" + T3Date.newDate(date); - result = lengthWeightConversions.get(key); + String key = species.getCode() + "-" + ocean.getCode() + "-" + sex + "-" + T3Date.newDate(date); + LengthWeightConversion result = lengthWeightConversions.get(key); if (result == null && !lengthWeightConversions.containsKey(key)) { // load it from db result = dao.findLengthWeightConversion(species, ocean, 0, date); @@ -140,17 +133,16 @@ public class LengthWeightConversionHelper { } /** - * Get the convertor for the given parameters from the cache. + * Get the converter for the given parameters from the cache. * <p/> * If not found, then load it from db. * * @param species species to use * @return convertor found - * @throws TopiaException if any db while querying db */ public LengthWeightConversion getConversions(Species species) { Objects.requireNonNull(conversionContext, - "No conversion context, must specify all parameters to obtain a convertor."); + "No conversion context, must specify all parameters to obtain a converter."); LengthWeightConversion result; String key = species.getCode() + conversionContext.getKey(); result = lengthWeightConversions.get(key); @@ -163,24 +155,24 @@ public class LengthWeightConversionHelper { } return result; } - - public Integer getSpecieHighestLengthClass(Species species, WeightCategory weightCategory) { - LengthWeightConversion conversions = getConversions(species); - Integer result; - if (conversions == null) { - result = null; - } else { - result = getSpecieHighestLengthClass(conversions, weightCategory); - } - return result; - } - - public int getSpecieHighestLengthClass(LengthWeightConversion conversion, WeightCategory weightCategory) { - String key = conversion.getTopiaId() + "-" + weightCategory.getTopiaId(); +// +// public Integer getSpecieHighestLengthClass(Species species, WeightCategory weightCategory) { +// LengthWeightConversion conversions = getConversions(species); +// Integer result; +// if (conversions == null) { +// result = null; +// } else { +// result = getSpecieHighestLengthClass(conversions, weightCategory); +// } +// return result; +// } + + public int getSpecieHighestLengthClass(LengthWeightConversion conversion, Integer max) { + String key = conversion.getTopiaId() + "-" + max/*weightCategory.getTopiaId()*/; Integer result = lengthClassMap.get(key); if (result == null) { // load it once for all - result = conversion.getSpecieHighestLengthClass(weightCategory.getMax()); + result = conversion.getSpecieHighestLengthClass(max /*weightCategory.getMax()*/); // store it once for all lengthClassMap.put(key, result); log.info(String.format("Cache lengthClassMap [%d] for %s", lengthClassMap.size(), key)); ===================================== t3-domain/src/main/java/fr/ird/t3/models/WeightCompositionAggregateModel.java ===================================== --- a/t3-domain/src/main/java/fr/ird/t3/models/WeightCompositionAggregateModel.java +++ b/t3-domain/src/main/java/fr/ird/t3/models/WeightCompositionAggregateModel.java @@ -85,8 +85,7 @@ public class WeightCompositionAggregateModel implements Closeable { } public void addModel(WeightCompositionAggregateModel modelToMerge) { - for (WeightCompositionModel compositionModel : - modelToMerge.getModel().values()) { + for (WeightCompositionModel compositionModel : modelToMerge.getModel().values()) { addModel(compositionModel); } } ===================================== t3-domain/src/test/java/fr/ird/t3/entities/reference/LengthWeightConversionHelperTest.java ===================================== --- a/t3-domain/src/test/java/fr/ird/t3/entities/reference/LengthWeightConversionHelperTest.java +++ b/t3-domain/src/test/java/fr/ird/t3/entities/reference/LengthWeightConversionHelperTest.java @@ -127,7 +127,7 @@ public class LengthWeightConversionHelperTest extends AbstractDatabaseTest { Assert.assertFalse(allCategories.isEmpty()); for (WeightCategoryTreatment category : allCategories) { - conversionHelper.getSpecieHighestLengthClass(conversion, category); + conversionHelper.getSpecieHighestLengthClass(conversion, category.getMax()); } Range<Integer> open = Range.open(0, 1000); View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/compare/00d3c99d6971d8f50e8bdf2935b671a… --- View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/compare/00d3c99d6971d8f50e8bdf2935b671a… You're receiving this email because of your account on gitlab.com.

1 0

[Git][ultreiaio/ird-t3][develop] [N1][N2][N3] Permettre la sélection mutliple d'océans (See #260) - correction…
by Tony CHEMIT 20 Mar '18

20 Mar '18

Tony CHEMIT pushed to branch develop at ultreiaio / ird-t3 Commits: 00d3c99d by Tony CHEMIT at 2018-03-20T14:37:44Z [N1][N2][N3] Permettre la sélection mutliple d'océans (See #260) - correction sur le N1.5 (plus un peu de nettoyage de code) - - - - - 3 changed files: - t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ExtrapolateSampleWeightToSetAction.java - t3-domain/src/main/java/fr/ird/t3/entities/reference/LengthWeightConversionHelper.java - t3-domain/src/test/java/fr/ird/t3/entities/reference/LengthWeightConversionHelperTest.java Changes: ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ExtrapolateSampleWeightToSetAction.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ExtrapolateSampleWeightToSetAction.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level1/ExtrapolateSampleWeightToSetAction.java @@ -38,7 +38,6 @@ import fr.ird.t3.entities.reference.Species; import fr.ird.t3.entities.reference.SpeciesTopiaDao; import fr.ird.t3.entities.reference.WeightCategories; import fr.ird.t3.entities.reference.WeightCategorySample; -import fr.ird.t3.entities.reference.WeightCategorySampleImpl; import fr.ird.t3.services.DecoratorService; import fr.ird.t3.services.ioc.InjectDAO; import org.apache.commons.logging.Log; @@ -46,6 +45,7 @@ import org.apache.commons.logging.LogFactory; import org.nuiton.util.TimeLog; import java.util.Collection; +import java.util.Date; import java.util.HashMap; import java.util.HashSet; import java.util.Map; @@ -54,7 +54,7 @@ import java.util.Set; import static org.nuiton.i18n.I18n.l; /** - * Extrapolate weigth from sample to their owing set. + * Extrapolate weight from sample to their owing set. * * @author Tony Chemit - dev(a)tchemit.fr * @since 1.0 @@ -69,16 +69,13 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { private LengthWeightConversionTopiaDao lengthWeightConversionDAO; @InjectDAO(entityType = SetSpeciesFrequency.class) private SetSpeciesFrequencyTopiaDao setSpeciesFrequencyDAO; - // @InjectDAO(entityType = WeightCategoryWellPlan.class) -// private WeightCategoryWellPlanTopiaDao weightCategoryWellPlanDAO; - // private WeightCategoryWellPlan categoryM10; -// private WeightCategoryWellPlan categoryP10; private LengthWeightConversionHelper conversionHelper; public ExtrapolateSampleWeightToSetAction() { super(Level1Step.EXTRAPOLATE_SAMPLE_WEIGHT_TO_SET); } + @SuppressWarnings("unused") public int getNbTreatedSets() { return nbTreatedSets; } @@ -87,6 +84,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { this.nbTreatedSets = nbTreatedSets; } + @SuppressWarnings("unused") public float getNbTreatedFishesInSamples() { return nbTreatedFishesInSamples; } @@ -95,6 +93,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { this.nbTreatedFishesInSamples = nbTreatedFishesInSamples; } + @SuppressWarnings("unused") public float getNbCreatedFishesInSetSpeciesFrequency() { return nbCreatedFishesInSetSpeciesFrequency; } @@ -106,10 +105,6 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { @Override protected void prepareAction() throws Exception { super.prepareAction(); -// // get the -10Kg category -// setCategoryM10(weightCategoryWellPlanDAO.forCodeEquals(1).findUnique()); -// // get the +10Kg category -// setCategoryP10(weightCategoryWellPlanDAO.forCodeEquals(2).findUnique()); conversionHelper = lengthWeightConversionDAO.newConversionHelper(); } @@ -137,21 +132,21 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { private void doExecuteTrip(Trip trip, Collection<Sample> samples) { incrementsProgression(); - //FIXME I replace ocean by trip.departureHarbour.ocean, check this is ok (See #260) - // get the length class +10kg limit - Map<Species, Integer> limitLengthClassBySpecie = getThredHoldPlus10ForSpecies(trip, samples, trip.getDepartureHarbour().getOcean()); Set<Activity> tripActivities = new HashSet<>(); // extrapolate for each sample set number to set for (Sample sample : samples) { long s0 = TimeLog.getTime(); incrementsProgression(); - String sampleStr = l(locale, "t3.level1.extrapolateSampleWeightToSet.sampleStr", - decorate(trip), sample.getSampleNumber()); + String sampleStr = l(locale, "t3.level1.extrapolateSampleWeightToSet.sampleStr", decorate(trip), sample.getSampleNumber()); float nb = sample.getTotalStandardiseSampleSpeciesFrequencyNumber(); addInfoMessage(l(locale, "t3.level1.extrapolateSampleWeightToSet.sample.nbFishes", sampleStr, nb)); nbTreatedFishesInSamples += nb; + Set<Species> species = SpeciesTopiaDao.getAllSpeciesFromSampleSpecies(samples); for (SampleSet sampleSet : sample.getSampleSet()) { Activity activity = sampleSet.getActivity(); + Ocean ocean = activity.getOcean(); + // get the length class +10kg limit + Map<Species, Integer> limitLengthClassBySpecie = getThresholdPlus10ForSpecies(species, ocean, activity.getRoute().getDate()); boolean added = tripActivities.add(activity); if (added) { // remove all stuff from activity @@ -240,7 +235,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { } /** - * Given a sample well, compute the {@link SampleWellSetWeight}, says : + * Given a sample well, compute the {@link SampleWellSetWeight}, says: * <ul> * <li>his total weight</<li> * <li>his weight of -10Kg category</<li> @@ -272,25 +267,25 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { * * @param sampleWellStr sampleSet decorate value * @param setWeight the sampleSet weight computed - * @param sampletWeight the sampleSet weight computed + * @param sampleWeight the sampleSet weight computed * @param useRfMinus10AndRfPlus10 flag to try to use rf-10 and rf+10 * @return the computed rf context for the given sample set */ - private RFContext computeRFContext(String sampleWellStr, SampleWellSetWeight setWeight, SampleWellSampleWeight sampletWeight, boolean useRfMinus10AndRfPlus10) { + private RFContext computeRFContext(String sampleWellStr, SampleWellSetWeight setWeight, SampleWellSampleWeight sampleWeight, boolean useRfMinus10AndRfPlus10) { float pondt = setWeight.getTotalWeight(); Float pondp = setWeight.getPlus10Weight(); Float pondm = setWeight.getMinus10Weight(); // compute rftot - float pdecht = sampletWeight.getTotalWeight(); + float pdecht = sampleWeight.getTotalWeight(); float rftot = pondt / pdecht; // compute rfMinus10 - Float pdechm = sampletWeight.getMinus10Weight(); + Float pdechm = sampleWeight.getMinus10Weight(); Float rfMinus10 = null; if (pdechm != null && pdechm > 0 && pondm != null) { rfMinus10 = pondm / pdechm; } // compute rfPlus10 - Float pdechp = sampletWeight.getPlus10Weight(); + Float pdechp = sampleWeight.getPlus10Weight(); Float rfPlus10 = null; if (pdechp != null && pdechp > 0 && pondp != null) { rfPlus10 = pondp / pdechp; @@ -310,7 +305,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { rfMinus10Status = RfUsageStatus.REJECTED_RF_NOT_DEFINED; } else { int rfMinus10MinNumber = getConfiguration().getRfMinus10MinNumber(); - Float sampletWeightMinus10Number = sampletWeight.getMinus10Number(); + Float sampletWeightMinus10Number = sampleWeight.getMinus10Number(); if (Math.abs(rfMinus10) < 0.001f) { addWarningMessage(l(locale, "t3.level1.extrapolateSampleWeightToSet.warning.rfMinus10.too.low", sampleWellStr, rfMinus10, rftot)); @@ -331,7 +326,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { rfPlus10Status = RfUsageStatus.REJECTED_RF_NOT_DEFINED; } else { int rfPlus10MinNumber = getConfiguration().getRfPlus10MinNumber(); - Float sampletWeightPlus10Number = sampletWeight.getPlus10Number(); + Float sampletWeightPlus10Number = sampleWeight.getPlus10Number(); if (Math.abs(rfPlus10) < 0.001f) { addWarningMessage(l(locale, "t3.level1.extrapolateSampleWeightToSet.warning.rfPlus10.too.low", sampleWellStr, rfPlus10, rftot)); @@ -387,25 +382,21 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { } /** - * Obtain for each species used in one of the given sample the first length - * class which represents a +10Kg weight. + * Obtain for each species used in one of the given sample the first length class which represents a +10Kg weight. * - * @param trip the trip where samples are done - * @param samples the sample to scan - * @param ocean ocean where trip happens + * @param species species to scan + * @param ocean where (used to get conversion) + * @param date when (used to get conversion) * @return the universe of length class limits computed indexed by species */ - private Map<Species, Integer> getThredHoldPlus10ForSpecies(Trip trip, Collection<Sample> samples, Ocean ocean) { - Set<Species> species = SpeciesTopiaDao.getAllSpeciesFromSampleSpecies(samples); + private Map<Species, Integer> getThresholdPlus10ForSpecies(Set<Species> species, Ocean ocean, Date date) { // for each of those species, found the length class which matches a weight > 10Kg Map<Species, Integer> result = new HashMap<>(); for (Species specie : species) { - LengthWeightConversion conversion = conversionHelper.getConversions(specie, ocean, 0, trip.getLandingDate()); + LengthWeightConversion conversion = conversionHelper.getConversions(specie, ocean, 0, date); if (conversion != null) { // only add conversion for species if found - WeightCategorySample fakeCategory = new WeightCategorySampleImpl(); - fakeCategory.setMax(10); - int lengthClass = conversionHelper.getSpecieHighestLengthClass(conversion, fakeCategory); + int lengthClass = conversionHelper.getSpecieHighestLengthClass(conversion, 10); result.put(specie, lengthClass); } } @@ -417,14 +408,6 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { return result; } -// private void setCategoryM10(WeightCategoryWellPlan categoryM10) { -// this.categoryM10 = categoryM10; -// } -// -// private void setCategoryP10(WeightCategoryWellPlan categoryP10) { -// this.categoryP10 = categoryP10; -// } - /** * This object contains the sample weights (-10,+10 and total) for sample well. * <ul> @@ -436,9 +419,9 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { */ protected static class SampleWellSampleWeight { - /** sum of fishes wieght for length class < +10Kg. */ + /** sum of fishes weight for length class < +10Kg. */ Float minus10Weight; - /** sum of fishes wieght for length class > +10Kg. */ + /** sum of fishes weight for length class > +10Kg. */ Float plus10Weight; /** Number of fishes for length class > -10Kg. */ private Float plus10Number; @@ -476,8 +459,7 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { } /** - * This object contains the set weights (-10,+10 and total) for - * sample well. + * This object contains the set weights (-10,+10 and total) for sample well. * <ul> * <li>{@link #getMinus10Weight()} is legacy {@code pondm}</li> * <li>{@link #getPlus10Weight()}} is legacy {@code pondp}</li> @@ -486,11 +468,11 @@ public class ExtrapolateSampleWeightToSetAction extends AbstractLevel1Action { */ protected static class SampleWellSetWeight { - /** sum of fishes wieght for length class < +10Kg. */ + /** sum of fishes weight for length class < +10Kg. */ Float totalWeight; - /** sum of fishes wieght for length class < +10Kg. */ + /** sum of fishes weight for length class < +10Kg. */ Float minus10Weight; - /** sum of fishes wieght for length class > +10Kg. */ + /** sum of fishes weight for length class > +10Kg. */ Float plus10Weight; protected static SampleWellSetWeight create(Float totalWeight, Float minus10Weight, Float plus10Weight) { ===================================== t3-domain/src/main/java/fr/ird/t3/entities/reference/LengthWeightConversionHelper.java ===================================== --- a/t3-domain/src/main/java/fr/ird/t3/entities/reference/LengthWeightConversionHelper.java +++ b/t3-domain/src/main/java/fr/ird/t3/entities/reference/LengthWeightConversionHelper.java @@ -40,28 +40,25 @@ import java.util.TreeMap; */ public class LengthWeightConversionHelper { - /** Logger. */ - private static final Log log = - LogFactory.getLog(LengthWeightConversionHelper.class); + private static final Log log = LogFactory.getLog(LengthWeightConversionHelper.class); - protected final LengthWeightConversionTopiaDao dao; + private final LengthWeightConversionTopiaDao dao; private final Map<String, LengthWeightConversion> lengthWeightConversions; private final Map<String, Integer> lengthClassMap; - private final ConversionContext conversionContext; LengthWeightConversionHelper(LengthWeightConversionTopiaDao dao, Ocean ocean, int sex, Date date) { this.dao = dao; this.conversionContext = new ConversionContext(ocean, sex, date); - lengthWeightConversions = new TreeMap<>(); - lengthClassMap = new TreeMap<>(); + this.lengthWeightConversions = new TreeMap<>(); + this.lengthClassMap = new TreeMap<>(); } LengthWeightConversionHelper(LengthWeightConversionTopiaDao dao) { this.dao = dao; this.conversionContext = null; - lengthWeightConversions = new TreeMap<>(); - lengthClassMap = new TreeMap<>(); + this.lengthWeightConversions = new TreeMap<>(); + this.lengthClassMap = new TreeMap<>(); } private <W extends WeightCategory> W getNextCategory(LengthWeightConversion conversion, int lengthClass, Iterator<W> itr) { @@ -77,7 +74,7 @@ public class LengthWeightConversionHelper { } else { // get max lengthClass - int nextLengthClass = getSpecieHighestLengthClass(conversion, result); + int nextLengthClass = getSpecieHighestLengthClass(conversion, result.getMax()); if (nextLengthClass < lengthClass) { @@ -89,10 +86,9 @@ public class LengthWeightConversionHelper { return result; } - public <W extends WeightCategory> Map<Integer, W> getWeightCategoriesDistribution( - LengthWeightConversion conversion, - List<W> weightCategories, - List<Integer> lengthClasses) { + public <W extends WeightCategory> Map<Integer, W> getWeightCategoriesDistribution(LengthWeightConversion conversion, + List<W> weightCategories, + List<Integer> lengthClasses) { Map<Integer, W> result = new TreeMap<>(); W currentWeightCategory = null; int currentMaxLengthClass = -1; @@ -105,7 +101,7 @@ public class LengthWeightConversionHelper { if (currentWeightCategory == null) { // get the correct weight category currentWeightCategory = getNextCategory(conversion, lengthClass, itr); - currentMaxLengthClass = getSpecieHighestLengthClass(conversion, currentWeightCategory); + currentMaxLengthClass = getSpecieHighestLengthClass(conversion, currentWeightCategory.getMax()); } result.put(lengthClass, currentWeightCategory); } @@ -113,7 +109,7 @@ public class LengthWeightConversionHelper { } /** - * Get the convertor for the given parameters from the cache. + * Get the converter for the given parameters from the cache. * <p/> * If not found, then load it from db. * @@ -122,13 +118,10 @@ public class LengthWeightConversionHelper { * @param sex sex to use * @param date min date to use * @return convertor found - * @throws TopiaException if any db while querying db */ public LengthWeightConversion getConversions(Species species, Ocean ocean, int sex, Date date) { - LengthWeightConversion result; - String key = species.getCode() + "-" + ocean.getCode() + - "-" + sex + "-" + T3Date.newDate(date); - result = lengthWeightConversions.get(key); + String key = species.getCode() + "-" + ocean.getCode() + "-" + sex + "-" + T3Date.newDate(date); + LengthWeightConversion result = lengthWeightConversions.get(key); if (result == null && !lengthWeightConversions.containsKey(key)) { // load it from db result = dao.findLengthWeightConversion(species, ocean, 0, date); @@ -140,17 +133,16 @@ public class LengthWeightConversionHelper { } /** - * Get the convertor for the given parameters from the cache. + * Get the converter for the given parameters from the cache. * <p/> * If not found, then load it from db. * * @param species species to use * @return convertor found - * @throws TopiaException if any db while querying db */ public LengthWeightConversion getConversions(Species species) { Objects.requireNonNull(conversionContext, - "No conversion context, must specify all parameters to obtain a convertor."); + "No conversion context, must specify all parameters to obtain a converter."); LengthWeightConversion result; String key = species.getCode() + conversionContext.getKey(); result = lengthWeightConversions.get(key); @@ -163,24 +155,24 @@ public class LengthWeightConversionHelper { } return result; } - - public Integer getSpecieHighestLengthClass(Species species, WeightCategory weightCategory) { - LengthWeightConversion conversions = getConversions(species); - Integer result; - if (conversions == null) { - result = null; - } else { - result = getSpecieHighestLengthClass(conversions, weightCategory); - } - return result; - } - - public int getSpecieHighestLengthClass(LengthWeightConversion conversion, WeightCategory weightCategory) { - String key = conversion.getTopiaId() + "-" + weightCategory.getTopiaId(); +// +// public Integer getSpecieHighestLengthClass(Species species, WeightCategory weightCategory) { +// LengthWeightConversion conversions = getConversions(species); +// Integer result; +// if (conversions == null) { +// result = null; +// } else { +// result = getSpecieHighestLengthClass(conversions, weightCategory); +// } +// return result; +// } + + public int getSpecieHighestLengthClass(LengthWeightConversion conversion, Integer max) { + String key = conversion.getTopiaId() + "-" + max/*weightCategory.getTopiaId()*/; Integer result = lengthClassMap.get(key); if (result == null) { // load it once for all - result = conversion.getSpecieHighestLengthClass(weightCategory.getMax()); + result = conversion.getSpecieHighestLengthClass(max /*weightCategory.getMax()*/); // store it once for all lengthClassMap.put(key, result); log.info(String.format("Cache lengthClassMap [%d] for %s", lengthClassMap.size(), key)); ===================================== t3-domain/src/test/java/fr/ird/t3/entities/reference/LengthWeightConversionHelperTest.java ===================================== --- a/t3-domain/src/test/java/fr/ird/t3/entities/reference/LengthWeightConversionHelperTest.java +++ b/t3-domain/src/test/java/fr/ird/t3/entities/reference/LengthWeightConversionHelperTest.java @@ -127,7 +127,7 @@ public class LengthWeightConversionHelperTest extends AbstractDatabaseTest { Assert.assertFalse(allCategories.isEmpty()); for (WeightCategoryTreatment category : allCategories) { - conversionHelper.getSpecieHighestLengthClass(conversion, category); + conversionHelper.getSpecieHighestLengthClass(conversion, category.getMax()); } Range<Integer> open = Range.open(0, 1000); View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/00d3c99d6971d8f50e8bdf2935b671ab… --- View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/00d3c99d6971d8f50e8bdf2935b671ab… You're receiving this email because of your account on gitlab.com.

1 0

[Git][ultreiaio/ird-t3][develop] [N2][N3] Gestion des catégories de poids -10/+10kg dans la stratification N2 et…
by Tony CHEMIT 20 Mar '18

20 Mar '18

Tony CHEMIT pushed to branch develop at ultreiaio / ird-t3 Commits: a8ec9c67 by Tony CHEMIT at 2018-03-20T13:14:06Z [N2][N3] Gestion des catégories de poids -10/+10kg dans la stratification N2 et N3 (See #263) - nettoyage de code et ajout de FIXME... - - - - - 8 changed files: - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratum.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratumLoader.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratum.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2Action.java - t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3SampleStratum.java - t3-actions/src/test/java/fr/ird/t3/actions/ActionResumeTest.java - t3-domain/src/main/java/fr/ird/t3/actions/stratum/SampleStratum.java - t3-domain/src/main/java/fr/ird/t3/models/WeightCompositionAggregateModel.java Changes: ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratum.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratum.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratum.java @@ -8,12 +8,12 @@ * it under the terms of the GNU Affero General Public License as published by * the Free Software Foundation, either version 3 of the License, or * (at your option) any later version. - * + * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. - * + * * You should have received a copy of the GNU Affero General Public License * along with this program. If not, see <http://www.gnu.org/licenses/>. * #L% @@ -37,6 +37,8 @@ import fr.ird.t3.models.WeightCompositionModel; import fr.ird.t3.models.WeightCompositionModelHelper; import fr.ird.t3.services.DecoratorService; import fr.ird.t3.services.T3ServiceContext; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; import java.io.IOException; import java.util.Collection; @@ -55,6 +57,7 @@ import static org.nuiton.i18n.I18n.l; */ public class L2CatchStratum extends CatchStratum<Level2Configuration, Level2Action> { + private static final Log log = LogFactory.getLog(L2CatchStratum.class); /** * All species used by all weight categories found in all catches * for this stratum. @@ -100,13 +103,38 @@ public class L2CatchStratum extends CatchStratum<Level2Configuration, Level2Acti */ private WeightCompositionAggregateModel inputModelForSpeciesToFix; - L2CatchStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, Collection<Species> speciesToFix) { + private L2CatchStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, Collection<Species> speciesToFix) { super(stratumConfiguration, speciesToFix); weightCategoriesForSpecies = HashMultimap.create(); inputModelForAllSpecies = new WeightCompositionAggregateModel(); outputModelForAllSpecies = new WeightCompositionAggregateModel(); } + static L2CatchStratum newCatchStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, + List<WeightCategoryTreatment> weightCategories, + Collection<Species> species, + Level2Action action) throws Exception { + L2CatchStratum catchStratum = new L2CatchStratum(stratumConfiguration, species); + T3ServiceContext serviceContext = action.getServiceContext(); + catchStratum.init(serviceContext, weightCategories, action); + // get the total weight of the catch stratum + float catchStratumWeight = catchStratum.getTotalCatchWeightForSpeciesToFix(); + if (catchStratumWeight == 0) { + // no catch in this stratum, skip it + String message = l(serviceContext.getLocale(), "t3.level2.message.noCatch.in.stratum"); + log.info(message); + action.addInfoMessage(message); + // let's nullify the catch stratum (make it no more available) + catchStratum = null; + } else { + // log it + String message = catchStratum.logCatchStratum(action.getDecoratorService()); + log.info(message); + action.addInfoMessage(message); + } + return catchStratum; + } + @Override public void close() throws IOException { super.close(); @@ -130,11 +158,13 @@ public class L2CatchStratum extends CatchStratum<Level2Configuration, Level2Acti Integer nbZones = e.getValue(); if (activity.isCorrectedElementaryCatchNotEmpty()) { + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie for (CorrectedElementaryCatch aCatch : activity.getCorrectedElementaryCatch()) { weightCategoriesForSpecies.put(aCatch.getWeightCategoryTreatment(), aCatch.getSpecies()); } } + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie Multimap<WeightCategoryTreatment, CorrectedElementaryCatch> correctedElementaryCatchesByCategory = ActivityTopiaDao.groupByWeightCategoryTreatment(activity.getCorrectedElementaryCatch()); ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratumLoader.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratumLoader.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2CatchStratumLoader.java @@ -25,8 +25,10 @@ import fr.ird.t3.actions.stratum.CatchStratumLoader; import fr.ird.t3.actions.stratum.StratumConfiguration; import fr.ird.t3.entities.data.Activity; import fr.ird.t3.entities.data.ActivityTopiaDao; +import fr.ird.t3.entities.data.CorrectedElementaryCatch; import fr.ird.t3.entities.data.Trip; import fr.ird.t3.entities.reference.Vessel; +import fr.ird.t3.entities.reference.WeightCategoryTreatment; import fr.ird.t3.services.ioc.InjectDAO; import java.util.HashMap; @@ -71,6 +73,7 @@ public class L2CatchStratumLoader extends CatchStratumLoader<Level2Configuration Map<Activity, Integer> result = new HashMap<>(); if (activityIds != null && activityIds.size() > 0) { Set<Vessel> possibleVessels = configuration.getPossibleCatchVessels(); + WeightCategoryTreatment weightCategoryTreatment = configuration.getWeightCategoryTreatment(); for (Map.Entry<String, Integer> e : activityIds.entrySet()) { String activityId = e.getKey(); // get activity @@ -84,6 +87,14 @@ public class L2CatchStratumLoader extends CatchStratumLoader<Level2Configuration // recheck activity have some catches. Preconditions.checkState(activity.isCorrectedElementaryCatchNotEmpty(), String.format("Can not accept an activity (%s) with no catch", activity.getTopiaId())); + if (weightCategoryTreatment != null) { + // weight category is in stratum, check there is catch using this category + boolean foundWeightCategory = activity.getCorrectedElementaryCatch().stream().map(CorrectedElementaryCatch::getWeightCategoryTreatment).anyMatch(c -> c.equals(weightCategoryTreatment)); + if (!foundWeightCategory) { + // reject - the stratum weight category not found for this activity + continue; + } + } result.put(activity, e.getValue()); } } ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratum.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratum.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/L2SampleStratum.java @@ -26,11 +26,12 @@ import fr.ird.t3.entities.data.Activity; import fr.ird.t3.entities.data.ActivityTopiaDao; import fr.ird.t3.entities.data.SetSpeciesFrequency; import fr.ird.t3.entities.reference.Species; +import fr.ird.t3.entities.reference.WeightCategory; import fr.ird.t3.entities.reference.WeightCategorySample; +import fr.ird.t3.entities.reference.WeightCategoryTreatment; import fr.ird.t3.models.WeightCompositionAggregateModel; import fr.ird.t3.models.WeightCompositionModelHelper; import fr.ird.t3.services.T3ServiceContext; -import org.apache.commons.collections.CollectionUtils; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.nuiton.topia.persistence.TopiaException; @@ -38,6 +39,7 @@ import org.nuiton.topia.persistence.TopiaException; import java.io.IOException; import java.util.Collection; import java.util.HashMap; +import java.util.List; import java.util.Locale; import java.util.Map; import java.util.Set; @@ -81,12 +83,22 @@ public class L2SampleStratum extends SampleStratum<Level2Configuration, Level2Ac */ private float sampleStratumTotalWeight; - L2SampleStratum(StratumConfiguration<Level2Configuration> configuration, Collection<Species> speciesToFix, float catchStratumTotalWeight) { + private L2SampleStratum(StratumConfiguration<Level2Configuration> configuration, Collection<Species> speciesToFix, float catchStratumTotalWeight) { super(configuration, speciesToFix); this.catchStratumTotalWeight = catchStratumTotalWeight; modelsForAllSpecies = new WeightCompositionAggregateModel(); } + static L2SampleStratum newSampleStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, + List<WeightCategoryTreatment> weightCategories, + float totalCatchWeight, + Collection<Species> species, + Level2Action action) throws Exception { + L2SampleStratum sampleStratum = new L2SampleStratum(stratumConfiguration, species, totalCatchWeight); + sampleStratum.init(action.getServiceContext(), weightCategories, action); + return sampleStratum; + } + @Override public void close() throws IOException { super.close(); @@ -97,18 +109,14 @@ public class L2SampleStratum extends SampleStratum<Level2Configuration, Level2Ac @Override protected L2SampleStratumLoader newLoader() { int oceanCode = getConfiguration().getZone().getOcean().getCode(); - L2SampleStratumLoader result; switch (oceanCode) { case 1: - result = new L2SampleStratumLoaderAtlantic(this); - break; + return new L2SampleStratumLoaderAtlantic(this); case 2: - result = new L2SampleStratumLoaderIndian(this); - break; + return new L2SampleStratumLoaderIndian(this); default: throw new IllegalStateException("Not implemented for ocean with code " + oceanCode); } - return result; } public WeightCompositionAggregateModel getModelsForSpeciesToFix() { @@ -137,7 +145,7 @@ public class L2SampleStratum extends SampleStratum<Level2Configuration, Level2Ac ActivityTopiaDao.fillWeightsFromSetSpeciesCatWeight(activity, weights, null); // obtain the set species frequencies for the current activity Collection<SetSpeciesFrequency> setSpeciesFrequencies = activity.getSetSpeciesFrequency(); - if (CollectionUtils.isNotEmpty(setSpeciesFrequencies)) { + if (activity.isSetSpeciesFrequencyNotEmpty()) { // compute sample count for this activity int newCount = computeSampleCount(setSpeciesFrequencies, species); // merge it with final total count @@ -145,24 +153,25 @@ public class L2SampleStratum extends SampleStratum<Level2Configuration, Level2Ac } } // add all weights to model - for (WeightCategorySample weightCategoryTreatment : weights.keySet()) { - Map<Species, Float> speciesFloatMap = weights.get(weightCategoryTreatment); - modelsForAllSpecies.addModel(weightCategoryTreatment, speciesFloatMap); + for (Map.Entry<WeightCategorySample, Map<Species, Float>> e : weights.entrySet()) { + Map<Species, Float> speciesFloatMap = e.getValue(); + WeightCategory weightCategorySample = e.getKey(); + modelsForAllSpecies.addModel(weightCategorySample, speciesFloatMap); } // recompute the weight model for species to fix modelsForSpeciesToFix = modelsForAllSpecies.extractForSpecies(species); // recompute the total sample weight (for species to fix) sampleStratumTotalWeight = modelsForSpeciesToFix.getTotalModel().getTotalWeight(); - addMergedActivitesCount(activities.size()); + addMergedActivitiesCount(activities.size()); log.info(String.format("sampleStratumTotalCount = %d / sampleStratumTotalWeight = %f", getSampleStratumTotalCount(), getSampleStratumTotalWeight())); } - private int computeSampleCount(Collection<SetSpeciesFrequency> newDatas, Collection<Species> speciesToFix) { + private int computeSampleCount(Collection<SetSpeciesFrequency> setSpeciesFrequencies, Collection<Species> speciesToFix) { int newCount = 0; - for (SetSpeciesFrequency newData : newDatas) { - Species species = newData.getSpecies(); + for (SetSpeciesFrequency setSpeciesFrequency : setSpeciesFrequencies) { + Species species = setSpeciesFrequency.getSpecies(); if (speciesToFix.contains(species)) { - newCount += newData.getNumber(); + newCount += setSpeciesFrequency.getNumber(); } } return newCount; ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2Action.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2Action.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level2/Level2Action.java @@ -51,6 +51,7 @@ import fr.ird.t3.models.WeightCompositionAggregateModel; import fr.ird.t3.models.WeightCompositionModel; import fr.ird.t3.models.WeightCompositionModelHelper; import fr.ird.t3.services.DecoratorService; +import fr.ird.t3.services.T3ServiceContext; import fr.ird.t3.services.ZoneStratumService; import fr.ird.t3.services.ioc.InjectDAO; import fr.ird.t3.services.ioc.InjectEntitiesById; @@ -113,7 +114,7 @@ public class Level2Action extends T3Action<Level2Configuration> { * * @since 1.3 */ - private Collection<L2StratumResult> stratumsResult; + private Collection<L2StratumResult> stratumResultSet; @InjectDAO(entityType = Activity.class) private ActivityTopiaDao activityDAO; @InjectDAO(entityType = WeightCategoryTreatment.class) @@ -179,7 +180,7 @@ public class Level2Action extends T3Action<Level2Configuration> { this.possibleCatchVessels = vesselDAO.getPossibleCatchVessels(catchFleets); // get possible vessels for sample stratum this.possibleSampleVessels = vesselDAO.getPossibleSampleVessels(sampleFleets, sampleFlags); - this.stratumsResult = new LinkedHashSet<>(); + this.stratumResultSet = new LinkedHashSet<>(); this.inputCatchModelForAllSpecies = new WeightCompositionAggregateModel(); this.outputCatchModelForAllSpecies = new WeightCompositionAggregateModel(); } @@ -278,7 +279,7 @@ public class Level2Action extends T3Action<Level2Configuration> { activityCache); try { L2StratumResult result = doExecuteStratum(stratumConfiguration, weightCategories, stratumIndex, usedActivityIds); - stratumsResult.add(result); + stratumResultSet.add(result); } finally { flushTransaction("After stratum " + stratumIndex); } @@ -319,7 +320,7 @@ public class Level2Action extends T3Action<Level2Configuration> { addInfoMessage(message); addInfoMessage("=============================================================================================="); // compute the catch stratum - try (L2CatchStratum catchStratum = newCatchStratum(stratumConfiguration, weightCategories)) { + try (L2CatchStratum catchStratum = L2CatchStratum.newCatchStratum(stratumConfiguration, weightCategories, species, this)) { incrementsProgression(); if (catchStratum == null) { // no catch in this stratum @@ -328,7 +329,7 @@ public class Level2Action extends T3Action<Level2Configuration> { incrementsProgression(); } else { // compute sample stratum - try (L2SampleStratum sampleStratum = newSampleStratum(stratumConfiguration, weightCategories, catchStratum.getTotalCatchWeightForSpeciesToFix())) { + try (L2SampleStratum sampleStratum = L2SampleStratum.newSampleStratum(stratumConfiguration, weightCategories, catchStratum.getTotalCatchWeightForSpeciesToFix(), species, this)) { incrementsProgression(); // get the substitution level for the sample stratum Integer level = sampleStratum.getSubstitutionLevel(); @@ -349,7 +350,7 @@ public class Level2Action extends T3Action<Level2Configuration> { // int nbZones = e.getValue(); // is activity was already treated ? boolean newActivity = usedActivityIds.add(activity.getTopiaId()); - doExecuteActivityInCatchStratum(catchStratum, + doExecuteActivityInCatchStratumWithoutSampleStratum(catchStratum, activity, activityIndex++, nbActivities, @@ -408,35 +409,104 @@ public class Level2Action extends T3Action<Level2Configuration> { private Collection<ZoneStratumAware> getZones(SchoolType schoolType) { return oceanContext.values().stream().flatMap(o -> o.getZones(schoolType).stream()).distinct().collect(Collectors.toList()); } +// +// private L2CatchStratum newCatchStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, +// List<WeightCategoryTreatment> weightCategories) throws Exception { +// L2CatchStratum catchStratum = new L2CatchStratum(stratumConfiguration, species); +// catchStratum.init(serviceContext, weightCategories, this); +// // get the total weight of the catch stratum +// float catchStratumWeight = catchStratum.getTotalCatchWeightForSpeciesToFix(); +// if (catchStratumWeight == 0) { +// // no catch in this stratum, skip it +// String message = l(locale, "t3.level2.message.noCatch.in.stratum"); +// log.info(message); +// addInfoMessage(message); +// // let's nullify the catch stratum (make it no more available) +// catchStratum = null; +// } else { +// // log it +// String message = catchStratum.logCatchStratum(getDecoratorService()); +// log.info(message); +// addInfoMessage(message); +// } +// return catchStratum; +// } + +// private L2SampleStratum newSampleStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, +// List<WeightCategoryTreatment> weightCategories, +// float totalCatchWeight) throws Exception { +// L2SampleStratum sampleStratum = new L2SampleStratum(stratumConfiguration, species, totalCatchWeight); +// sampleStratum.init(serviceContext, weightCategories, this); +// return sampleStratum; +// } + + private void doExecuteActivityInCatchStratumWithoutSampleStratum(L2CatchStratum catchStratum, + Activity activity, + int activityIndex, + int nbActivities, + boolean deleteOldData) { - private L2CatchStratum newCatchStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, - List<WeightCategoryTreatment> weightCategories) throws Exception { - L2CatchStratum catchStratum = new L2CatchStratum(stratumConfiguration, species); - catchStratum.init(serviceContext, weightCategories, this); - // get the total weight of the catch stratum - float catchStratumWeight = catchStratum.getTotalCatchWeightForSpeciesToFix(); - if (catchStratumWeight == 0) { - // no catch in this stratum, skip it - String message = l(locale, "t3.level2.message.noCatch.in.stratum"); - log.info(message); - addInfoMessage(message); - // let's nullify the catch stratum (make it no more available) - catchStratum = null; - } else { - // log it - String message = catchStratum.logCatchStratum(getDecoratorService()); - log.info(message); - addInfoMessage(message); + String activityStr = String.format("%s (%s)", decorate(activity), decorate(activity.getTrip(), DecoratorService.WITH_ID)); + String message = l(locale, "t3.level2.message.start.activity", activityIndex, nbActivities, activityStr, 1); + log.info(message); + addInfoMessage(message); + if (deleteOldData) { + // delete old data for this activity + log.info(String.format("Delete previous level2 data of %s", activityStr)); + activity.deleteComputedDataLevel2(); + } + // build a model for getting total of each weight category + try (WeightCompositionAggregateModel catchWeightModelForAllSpecies = new WeightCompositionAggregateModel()) { + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie + ActivityTopiaDao.fillWeightsFromCatchesWeight(activity, catchWeightModelForAllSpecies, 1); + try (WeightCompositionAggregateModel catchWeightModelForSpeciesToFix = catchWeightModelForAllSpecies.extractForSpecies(species)) { + // log catches weight + message = logCatchWeight(getDecoratorService(), catchWeightModelForAllSpecies, catchWeightModelForSpeciesToFix); + log.info(message); + addInfoMessage(message); + } + try (WeightCompositionAggregateModel model = new WeightCompositionAggregateModel()) { + ActivityTopiaDao.fillWeightsFromSetSpeciesCatWeight(activity, null, model); + try (WeightCompositionAggregateModel compositionModel = model.extractForSpecies(species)) { + String activityResume = logActivityCatchStratum(model, compositionModel, getDecoratorService()); + message = l(locale, "t3.level2.message.activity.with.sample.useOwn.composition", activityResume); + log.info(message); + addInfoMessage(message); + } + } + // apply composition model to activity catches + applySampleSpecificCompositionWithoutSampleStratum(catchStratum, activity); } - return catchStratum; } - private L2SampleStratum newSampleStratum(StratumConfiguration<Level2Configuration> stratumConfiguration, - List<WeightCategoryTreatment> weightCategories, - float totalCatchWeight) throws Exception { - L2SampleStratum sampleStratum = new L2SampleStratum(stratumConfiguration, species, totalCatchWeight); - sampleStratum.init(serviceContext, weightCategories, this); - return sampleStratum; + private void applySampleSpecificCompositionWithoutSampleStratum(L2CatchStratum catchStratum, Activity activity) { + try (WeightCompositionAggregateModel correctedCatchWeightModel = new WeightCompositionAggregateModel()) { + // Get all corrected catch group by weight category + Multimap<WeightCategoryTreatment, CorrectedElementaryCatch> correctedElementaryCatches = + ActivityTopiaDao.groupByWeightCategoryTreatment(activity.getCorrectedElementaryCatch()); + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie + for (WeightCategoryTreatment weightCategory : correctedElementaryCatches.keySet()) { + Map<Species, Float> weights = new HashMap<>(); + // Apply on all existing corrected catch weight + for (CorrectedElementaryCatch aCatch : correctedElementaryCatches.get(weightCategory)) { + Species speciesToUse = aCatch.getSpecies(); + // the corrected catch weight to add to the row + float correctedCatchWeight = aCatch.getCatchWeight(); + // hold added value (for logs) + weights.put(speciesToUse, correctedCatchWeight); + aCatch.setCorrectedCatchWeight(correctedCatchWeight); + aCatch.setCorrectedFlag(false); + } + // add corrected weights used for this category + correctedCatchWeightModel.addModel(weightCategory, weights); + } + // log corrected catches weight + String message = logCorrectedCatchWeight(correctedCatchWeightModel, getDecoratorService()); + log.info(message); + addInfoMessage(message); + // add corrected catch weight done for this activity in stratum output model + catchStratum.addActivityOutputModel(correctedCatchWeightModel); + } } private void doExecuteActivityInCatchStratum(L2CatchStratum catchStratum, @@ -457,14 +527,14 @@ public class Level2Action extends T3Action<Level2Configuration> { } // build a model for getting total of each weight category WeightCompositionAggregateModel catchWeightModelForAllSpecies = new WeightCompositionAggregateModel(); + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie ActivityTopiaDao.fillWeightsFromCatchesWeight(activity, catchWeightModelForAllSpecies, nbZones); WeightCompositionAggregateModel catchWeightModelForSpeciesToFix = catchWeightModelForAllSpecies.extractForSpecies(species); // log catches weight message = logCatchWeight(getDecoratorService(), catchWeightModelForAllSpecies, catchWeightModelForSpeciesToFix); log.info(message); addInfoMessage(message); - // obtain the sample composition model to use for the activity (can come from catch if it has some samples, - // otherwise use the catch stratum one) + // obtain the sample composition model to use for the activity (can come from catch if it has some samples, otherwise use the catch stratum one) boolean activityWithSample = catchStratum.isActivityWithSample(activity); boolean useAllSamplesOfStratum = getConfiguration().isUseAllSamplesOfStratum(); WeightCompositionAggregateModel compositionModel; @@ -498,161 +568,98 @@ public class Level2Action extends T3Action<Level2Configuration> { } } - private void doExecuteActivityInCatchStratum(L2CatchStratum catchStratum, - Activity activity, - int activityIndex, - int nbActivities, - boolean deleteOldData) { - - String activityStr = String.format("%s (%s)", decorate(activity), decorate(activity.getTrip(), DecoratorService.WITH_ID)); - String message = l(locale, "t3.level2.message.start.activity", activityIndex, nbActivities, activityStr, 1); - log.info(message); - addInfoMessage(message); - if (deleteOldData) { - // delete old data for this activity - log.info(String.format("Delete previous level2 data of %s", activityStr)); - activity.deleteComputedDataLevel2(); - } - // build a model for getting total of each weight category - WeightCompositionAggregateModel catchWeightModelForAllSpecies = new WeightCompositionAggregateModel(); - ActivityTopiaDao.fillWeightsFromCatchesWeight(activity, catchWeightModelForAllSpecies, 1); - WeightCompositionAggregateModel catchWeightModelForSpeciesToFix = catchWeightModelForAllSpecies.extractForSpecies(species); - // log catches weight - message = logCatchWeight(getDecoratorService(), catchWeightModelForAllSpecies, catchWeightModelForSpeciesToFix); - log.info(message); - addInfoMessage(message); - WeightCompositionAggregateModel model = new WeightCompositionAggregateModel(); - ActivityTopiaDao.fillWeightsFromSetSpeciesCatWeight(activity, null, model); - WeightCompositionAggregateModel compositionModel = model.extractForSpecies(species); - String activityResume = logActivityCatchStratum(model, compositionModel, getDecoratorService()); - message = l(locale, "t3.level2.message.activity.with.sample.useOwn.composition", activityResume); - log.info(message); - addInfoMessage(message); - // apply composition model to activity catches - applySampleSpecificComposition(catchStratum, activity); - catchWeightModelForAllSpecies.close(); - catchWeightModelForSpeciesToFix.close(); - compositionModel.close(); - } - - private void applySampleSpecificComposition(L2CatchStratum catchStratum, Activity activity) { - WeightCompositionAggregateModel correctedCatchWeightModel = new WeightCompositionAggregateModel(); - // Get all corrected catch group by weight category - Multimap<WeightCategoryTreatment, CorrectedElementaryCatch> correctedElementaryCatches = - ActivityTopiaDao.groupByWeightCategoryTreatment(activity.getCorrectedElementaryCatch()); - for (WeightCategoryTreatment weightCategory : correctedElementaryCatches.keySet()) { - Map<Species, Float> weights = new HashMap<>(); - // Apply on all existing corrected catch weight - for (CorrectedElementaryCatch aCatch : correctedElementaryCatches.get(weightCategory)) { - Species speciesToUse = aCatch.getSpecies(); - // the corrected catch weight to add to the row - float correctedCatchWeight = aCatch.getCatchWeight(); - // hold added value (for logs) - weights.put(speciesToUse, correctedCatchWeight); - aCatch.setCorrectedCatchWeight(correctedCatchWeight); - aCatch.setCorrectedFlag(false); - } - // add corrected weights used for this category - correctedCatchWeightModel.addModel(weightCategory, weights); - } - // log corrected catches weight - String message = logCorrectedCatchWeight(correctedCatchWeightModel, getDecoratorService()); - log.info(message); - addInfoMessage(message); - // add corrected catch weight done for this activity in stratum output model - catchStratum.addActivityOutputModel(correctedCatchWeightModel); - } - private void applySampleSpecificComposition(L2CatchStratum catchStratum, Activity activity, int nbZones, WeightCompositionAggregateModel sampleCompositionModel, WeightCompositionAggregateModel correctedCatchesForSpeciesToFix) { - WeightCompositionAggregateModel correctedCatchWeightModel = new WeightCompositionAggregateModel(); - // Get all corrected catch group by weight category - Multimap<WeightCategoryTreatment, CorrectedElementaryCatch> correctedElementaryCatches = - ActivityTopiaDao.groupByWeightCategoryTreatment(activity.getCorrectedElementaryCatch()); - for (WeightCategoryTreatment weightCategory : correctedElementaryCatches.keySet()) { - Map<Species, Float> weights = new HashMap<>(); - // get composition model to use - WeightCompositionModel model = sampleCompositionModel.getModel(weightCategory); - // set of species still to fix - Set<Species> speciesToFix; - // set of species still to fix - Set<Species> speciesStillToFix; - // get total weight for species to fix from correctedCatches - float totalWeight; - if (model == null) { - // this means there is no specific composition for this weight category to apply for species to fix - // says there is no species to fix (so all catches will be copied to correctedCatchWeight (with no fixedFlag) - speciesToFix = speciesStillToFix = Collections.emptySet(); - totalWeight = 0f; - } else { - // found a specific composition to apply for this weight category - speciesToFix = model.getSpecies(); - speciesStillToFix = new HashSet<>(speciesToFix); - totalWeight = correctedCatchesForSpeciesToFix.getModel(weightCategory).getTotalWeight(); - } - // divide the total weight to used by the number of zones of this activity - totalWeight = totalWeight / nbZones; - // Apply on all existing corrected catch weight - for (CorrectedElementaryCatch aCatch : correctedElementaryCatches.get(weightCategory)) { - Species speciesToUse = Objects.requireNonNull(aCatch.getSpecies()); - // flag (was fixed or not) - boolean toFix; - // the corrected catch weight to add to the row - float correctedCatchWeight; - if (model != null && speciesToFix.contains(speciesToUse)) { - // species to fix - toFix = true; - float weightRate = model.getWeightRate(speciesToUse); - // new corrected catch weight - correctedCatchWeight = totalWeight * weightRate; - // species no more to treat - speciesStillToFix.remove(speciesToUse); + try (WeightCompositionAggregateModel correctedCatchWeightModel = new WeightCompositionAggregateModel()) { + //FIXME-263 si la categorie de poids est dans la strate alors on ne doit que traiter cette catégorie + // Get all corrected catch group by weight category + Multimap<WeightCategoryTreatment, CorrectedElementaryCatch> correctedElementaryCatches = + ActivityTopiaDao.groupByWeightCategoryTreatment(activity.getCorrectedElementaryCatch()); + for (WeightCategoryTreatment weightCategory : correctedElementaryCatches.keySet()) { + Map<Species, Float> weights = new HashMap<>(); + // get composition model to use + WeightCompositionModel model = sampleCompositionModel.getModel(weightCategory); + // set of species still to fix + Set<Species> speciesToFix; + // set of species still to fix + Set<Species> speciesStillToFix; + // get total weight for species to fix from correctedCatches + float totalWeight; + if (model == null) { + // this means there is no specific composition for this weight category to apply for species to fix + // says there is no species to fix (so all catches will be copied to correctedCatchWeight (with no fixedFlag) + speciesToFix = speciesStillToFix = Collections.emptySet(); + totalWeight = 0f; } else { - // nothing to fix, just propagate old value - toFix = false; - correctedCatchWeight = aCatch.getCatchWeight(); + // found a specific composition to apply for this weight category + speciesToFix = model.getSpecies(); + speciesStillToFix = new HashSet<>(speciesToFix); + totalWeight = correctedCatchesForSpeciesToFix.getModel(weightCategory).getTotalWeight(); } - // hold added value (for logs) - weights.put(speciesToUse, correctedCatchWeight); - // Add to old value (if any) - Float oldCorrectedCatchWeight = aCatch.getCorrectedCatchWeight(); - if (oldCorrectedCatchWeight == null) { - oldCorrectedCatchWeight = 0f; + // divide the total weight to used by the number of zones of this activity + totalWeight = totalWeight / nbZones; + // Apply on all existing corrected catch weight + for (CorrectedElementaryCatch aCatch : correctedElementaryCatches.get(weightCategory)) { + Species speciesToUse = Objects.requireNonNull(aCatch.getSpecies()); + // flag (was fixed or not) + boolean toFix; + // the corrected catch weight to add to the row + float correctedCatchWeight; + if (model != null && speciesToFix.contains(speciesToUse)) { + // species to fix + toFix = true; + float weightRate = model.getWeightRate(speciesToUse); + // new corrected catch weight + correctedCatchWeight = totalWeight * weightRate; + // species no more to treat + speciesStillToFix.remove(speciesToUse); + } else { + // nothing to fix, just propagate old value + toFix = false; + correctedCatchWeight = aCatch.getCatchWeight(); + } + // hold added value (for logs) + weights.put(speciesToUse, correctedCatchWeight); + // Add to old value (if any) + Float oldCorrectedCatchWeight = aCatch.getCorrectedCatchWeight(); + if (oldCorrectedCatchWeight == null) { + oldCorrectedCatchWeight = 0f; + } + aCatch.setCorrectedCatchWeight(oldCorrectedCatchWeight + correctedCatchWeight); + aCatch.setCorrectedFlag(toFix); } - aCatch.setCorrectedCatchWeight(oldCorrectedCatchWeight + correctedCatchWeight); - aCatch.setCorrectedFlag(toFix); - } - // Creates new corrected catch weight records for all species still to fix - // Means they are in samples but not in catches - for (Species speciesToUse : speciesStillToFix) { - // weight rate of the species - float weightRate = model == null ? 1f : model.getWeightRate(speciesToUse); - // corrected catch weight - float correctedCatchWeight = totalWeight * weightRate; - // create new record - CorrectedElementaryCatch aCatch = correctedElementaryCatchDAO.create(); - aCatch.setSpecies(speciesToUse); - aCatch.setWeightCategoryTreatment(weightCategory); - aCatch.setCatchWeight(0f); - aCatch.setCorrectedFlag(true); - aCatch.setCorrectedCatchWeight(correctedCatchWeight); - // hold added value (for logs) - weights.put(speciesToUse, correctedCatchWeight); - // add it to activity - activity.addCorrectedElementaryCatch(aCatch); + // Creates new corrected catch weight records for all species still to fix + // Means they are in samples but not in catches + for (Species speciesToUse : speciesStillToFix) { + // weight rate of the species + float weightRate = model == null ? 1f : model.getWeightRate(speciesToUse); + // corrected catch weight + float correctedCatchWeight = totalWeight * weightRate; + // create new record + CorrectedElementaryCatch aCatch = correctedElementaryCatchDAO.create(); + aCatch.setSpecies(speciesToUse); + aCatch.setWeightCategoryTreatment(weightCategory); + aCatch.setCatchWeight(0f); + aCatch.setCorrectedFlag(true); + aCatch.setCorrectedCatchWeight(correctedCatchWeight); + // hold added value (for logs) + weights.put(speciesToUse, correctedCatchWeight); + // add it to activity + activity.addCorrectedElementaryCatch(aCatch); + } + // add corrected weights used for this category + correctedCatchWeightModel.addModel(weightCategory, weights); } - // add corrected weights used for this category - correctedCatchWeightModel.addModel(weightCategory, weights); + // log corrected catches weight + String message = logCorrectedCatchWeight(correctedCatchWeightModel, getDecoratorService()); + log.info(message); + addInfoMessage(message); + // add corrected catch weight done for this activity in stratum output model + catchStratum.addActivityOutputModel(correctedCatchWeightModel); } - // log corrected catches weight - String message = logCorrectedCatchWeight(correctedCatchWeightModel, getDecoratorService()); - log.info(message); - addInfoMessage(message); - // add corrected catch weight done for this activity in stratum output model - catchStratum.addActivityOutputModel(correctedCatchWeightModel); } // ------------------------------------------------------------------------- @@ -681,13 +688,13 @@ public class Level2Action extends T3Action<Level2Configuration> { @SuppressWarnings("unused") public int getNbStrataFixed() { - return stratumsResult.size(); + return stratumResultSet.size(); } @SuppressWarnings({"unused", "WeakerAccess"}) public Integer[] getAllSubstitutionLevels() { Set<Integer> levels = new HashSet<>(); - for (L2StratumResult stratumResult : stratumsResult) { + for (L2StratumResult stratumResult : stratumResultSet) { levels.add(stratumResult.getSubstitutionLevel()); } List<Integer> result = new ArrayList<>(levels); @@ -703,7 +710,7 @@ public class Level2Action extends T3Action<Level2Configuration> { @SuppressWarnings("unused") public Collection<L2StratumResult> getStratumResult(int level) { Set<L2StratumResult> singleResult = new LinkedHashSet<>(); - for (L2StratumResult stratumResult : stratumsResult) { + for (L2StratumResult stratumResult : stratumResultSet) { if (level == stratumResult.getSubstitutionLevel()) { singleResult.add(stratumResult); } @@ -717,7 +724,7 @@ public class Level2Action extends T3Action<Level2Configuration> { Integer[] levels = getAllSubstitutionLevels(); for (Integer level : levels) { Set<L2StratumResult> singleResult = new LinkedHashSet<>(); - for (L2StratumResult stratumResult : stratumsResult) { + for (L2StratumResult stratumResult : stratumResultSet) { if (level == stratumResult.getSubstitutionLevel()) { singleResult.add(stratumResult); } @@ -752,7 +759,7 @@ public class Level2Action extends T3Action<Level2Configuration> { @SuppressWarnings("unused") public int getMaximumSizeForStratum() { int result = 0; - for (L2StratumResult stratumResult : stratumsResult) { + for (L2StratumResult stratumResult : stratumResultSet) { result = Math.max(result, stratumResult.getLibelle().length() + 1); } return result; @@ -796,8 +803,8 @@ public class Level2Action extends T3Action<Level2Configuration> { modelForSpeciestoFix); } - public void setStratumsResult(Set<L2StratumResult> stratumsResult) { - this.stratumsResult = stratumsResult; + public void setStratumResultSet(Set<L2StratumResult> stratumResultSet) { + this.stratumResultSet = stratumResultSet; } public void setInputCatchModelForAllSpecies(WeightCompositionAggregateModel inputCatchModelForAllSpecies) { @@ -807,4 +814,8 @@ public class Level2Action extends T3Action<Level2Configuration> { public void setOutputCatchModelForAllSpecies(WeightCompositionAggregateModel outputCatchModelForAllSpecies) { this.outputCatchModelForAllSpecies = outputCatchModelForAllSpecies; } + + T3ServiceContext getServiceContext() { + return serviceContext; + } } ===================================== t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3SampleStratum.java ===================================== --- a/t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3SampleStratum.java +++ b/t3-actions/src/main/java/fr/ird/t3/actions/data/level3/L3SampleStratum.java @@ -181,7 +181,7 @@ public class L3SampleStratum extends SampleStratum<Level3Configuration, Level3Ac speciesCount.put(species, oldCount + newCount); } } - addMergedActivitesCount(activities.size()); + addMergedActivitiesCount(activities.size()); if (log.isInfoEnabled()) { log.info(String.format("sampleStratumTotalWeight = %s", getSampleStratumTotalWeight())); ===================================== t3-actions/src/test/java/fr/ird/t3/actions/ActionResumeTest.java ===================================== --- a/t3-actions/src/test/java/fr/ird/t3/actions/ActionResumeTest.java +++ b/t3-actions/src/test/java/fr/ird/t3/actions/ActionResumeTest.java @@ -746,7 +746,7 @@ public class ActionResumeTest<C extends T3ActionConfiguration, A extends T3Actio stratumResult.setNbActivities(5); stratumsResult.add(stratumResult); - action.setStratumsResult(stratumsResult); + action.setStratumResultSet(stratumsResult); action.setInputCatchModelForAllSpecies(new WeightCompositionAggregateModel()); action.setOutputCatchModelForAllSpecies(new WeightCompositionAggregateModel()); ===================================== t3-domain/src/main/java/fr/ird/t3/actions/stratum/SampleStratum.java ===================================== --- a/t3-domain/src/main/java/fr/ird/t3/actions/stratum/SampleStratum.java +++ b/t3-domain/src/main/java/fr/ird/t3/actions/stratum/SampleStratum.java @@ -27,7 +27,6 @@ import fr.ird.t3.entities.reference.WeightCategoryTreatment; import fr.ird.t3.services.IOCService; import fr.ird.t3.services.T3ServiceContext; import fr.ird.t3.services.ioc.InjectDAO; -import org.nuiton.topia.persistence.TopiaException; import java.util.Collection; import java.util.List; @@ -60,10 +59,13 @@ public abstract class SampleStratum<C extends LevelConfigurationWithStratum, A e */ private Integer substitutionLevel; + protected SampleStratum(StratumConfiguration<C> configuration, Collection<Species> speciesToFix) { + super(configuration, speciesToFix); + } + protected abstract SampleStratumLoader<C, A, S> newLoader(); - protected abstract String logSampleStratumLevel(int substitutionLevel, - A messager); + protected abstract String logSampleStratumLevel(int substitutionLevel, A messager); /** * Merge the given {@code activities} sample data in the stratum result. @@ -81,24 +83,15 @@ public abstract class SampleStratum<C extends LevelConfigurationWithStratum, A e * * @param serviceContext service context * @param activities the activities to merge - * @throws TopiaException if any database problem while loading data */ - protected abstract void mergeNewActivities(T3ServiceContext serviceContext, - Set<Activity> activities) throws TopiaException; - - protected SampleStratum(StratumConfiguration<C> configuration, - Collection<Species> speciesToFix) { - super(configuration, speciesToFix); - } + protected abstract void mergeNewActivities(T3ServiceContext serviceContext, Set<Activity> activities); - protected final void addMergedActivitesCount(int nb) { + protected final void addMergedActivitiesCount(int nb) { nbMergedActivities += nb; } @Override - public void init(T3ServiceContext serviceContext, - List<WeightCategoryTreatment> weightCategories, - A messager) throws Exception { + public void init(T3ServiceContext serviceContext, List<WeightCategoryTreatment> weightCategories, A messager) throws Exception { SampleStratumLoader<C, A, S> stratumLoader = newLoader(); ===================================== t3-domain/src/main/java/fr/ird/t3/models/WeightCompositionAggregateModel.java ===================================== --- a/t3-domain/src/main/java/fr/ird/t3/models/WeightCompositionAggregateModel.java +++ b/t3-domain/src/main/java/fr/ird/t3/models/WeightCompositionAggregateModel.java @@ -85,8 +85,7 @@ public class WeightCompositionAggregateModel implements Closeable { } public void addModel(WeightCompositionAggregateModel modelToMerge) { - for (WeightCompositionModel compositionModel : - modelToMerge.getModel().values()) { + for (WeightCompositionModel compositionModel : modelToMerge.getModel().values()) { addModel(compositionModel); } } View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/a8ec9c67461949609c21522a8c4e9601… --- View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/a8ec9c67461949609c21522a8c4e9601… You're receiving this email because of your account on gitlab.com.

1 0

[Git][ultreiaio/ird-t3][develop] [N0.3] Conversion des catégories logbook - rendre générique la codification des…
by Tony CHEMIT 20 Mar '18

20 Mar '18

Tony CHEMIT pushed to branch develop at ultreiaio / ird-t3 Commits: 07250611 by Tony CHEMIT at 2018-03-20T09:27:44Z [N0.3] Conversion des catégories logbook - rendre générique la codification des catégories inconnues - utilisation version par défaut (See #287) - - - - - 1 changed file: - t3-web/src/main/java/fr/ird/t3/web/actions/data/level0/ConvertCatchesWeightCategoriesConfigureAction.java Changes: ===================================== t3-web/src/main/java/fr/ird/t3/web/actions/data/level0/ConvertCatchesWeightCategoriesConfigureAction.java ===================================== --- a/t3-web/src/main/java/fr/ird/t3/web/actions/data/level0/ConvertCatchesWeightCategoriesConfigureAction.java +++ b/t3-web/src/main/java/fr/ird/t3/web/actions/data/level0/ConvertCatchesWeightCategoriesConfigureAction.java @@ -50,8 +50,12 @@ public class ConvertCatchesWeightCategoriesConfigureAction extends AbstractLevel protected void loadDefaultConfiguration(ConvertCatchesWeightCategoriesConfiguration config) { super.loadDefaultConfiguration(config); List<String> versions = dao.findAll().stream().map(WeightCategoryLogBookConversion::getVersion).distinct().collect(Collectors.toCollection(LinkedList::new)); + if (versions.isEmpty()) { + throw new IllegalStateException("can't find a version of weight category logBook conversion in database"); + } + String defaultVersion = versions.get(0); versions.add(ConvertCatchesWeightCategoriesConfiguration.LEGACY_VERSION); config.setVersions(versions); - config.setVersion(ConvertCatchesWeightCategoriesConfiguration.LEGACY_VERSION); + config.setVersion(defaultVersion); } } View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/0725061177b78eb4ca3fcbbaa2312ca2… --- View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/0725061177b78eb4ca3fcbbaa2312ca2… You're receiving this email because of your account on gitlab.com.

1 0

[Git][ultreiaio/ird-t3][develop] go to version 3.0
by Tony CHEMIT 19 Mar '18

19 Mar '18

Tony CHEMIT pushed to branch develop at ultreiaio / ird-t3 Commits: 985fcfc6 by Tony CHEMIT at 2018-03-19T22:55:48Z go to version 3.0 - - - - - 11 changed files: - pom.xml - t3-actions/pom.xml - t3-domain/pom.xml - t3-input-avdthv33/pom.xml - t3-input-avdthv35/pom.xml - t3-input-avdthv36/pom.xml - t3-installer/pom.xml - t3-msaccess-importer/pom.xml - t3-output-balbayav32/pom.xml - t3-web/pom.xml - t3/pom.xml Changes: ===================================== pom.xml ===================================== --- a/pom.xml +++ b/pom.xml @@ -28,7 +28,7 @@ <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> <packaging>pom</packaging> <name>T3 :: Pom</name> ===================================== t3-actions/pom.xml ===================================== --- a/t3-actions/pom.xml +++ b/t3-actions/pom.xml @@ -24,7 +24,7 @@ <parent> <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> </parent> <artifactId>t3-actions</artifactId> ===================================== t3-domain/pom.xml ===================================== --- a/t3-domain/pom.xml +++ b/t3-domain/pom.xml @@ -24,7 +24,7 @@ <parent> <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> </parent> <artifactId>t3-domain</artifactId> ===================================== t3-input-avdthv33/pom.xml ===================================== --- a/t3-input-avdthv33/pom.xml +++ b/t3-input-avdthv33/pom.xml @@ -24,7 +24,7 @@ <parent> <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> </parent> <artifactId>t3-input-avdthv33</artifactId> ===================================== t3-input-avdthv35/pom.xml ===================================== --- a/t3-input-avdthv35/pom.xml +++ b/t3-input-avdthv35/pom.xml @@ -24,7 +24,7 @@ <parent> <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> </parent> <artifactId>t3-input-avdthv35</artifactId> ===================================== t3-input-avdthv36/pom.xml ===================================== --- a/t3-input-avdthv36/pom.xml +++ b/t3-input-avdthv36/pom.xml @@ -24,7 +24,7 @@ <parent> <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> </parent> <artifactId>t3-input-avdthv36</artifactId> ===================================== t3-installer/pom.xml ===================================== --- a/t3-installer/pom.xml +++ b/t3-installer/pom.xml @@ -24,7 +24,7 @@ <parent> <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> </parent> <artifactId>t3-installer</artifactId> ===================================== t3-msaccess-importer/pom.xml ===================================== --- a/t3-msaccess-importer/pom.xml +++ b/t3-msaccess-importer/pom.xml @@ -24,7 +24,7 @@ <parent> <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> </parent> <artifactId>t3-msaccess-importer</artifactId> ===================================== t3-output-balbayav32/pom.xml ===================================== --- a/t3-output-balbayav32/pom.xml +++ b/t3-output-balbayav32/pom.xml @@ -24,7 +24,7 @@ <parent> <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> </parent> <artifactId>t3-output-balbayav32</artifactId> ===================================== t3-web/pom.xml ===================================== --- a/t3-web/pom.xml +++ b/t3-web/pom.xml @@ -24,7 +24,7 @@ <parent> <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> </parent> <artifactId>t3-web</artifactId> ===================================== t3/pom.xml ===================================== --- a/t3/pom.xml +++ b/t3/pom.xml @@ -26,7 +26,7 @@ <parent> <groupId>fr.ird.t3</groupId> <artifactId>ird-t3</artifactId> - <version>2.5-SNAPSHOT</version> + <version>3.0-SNAPSHOT</version> </parent> <artifactId>t3</artifactId> View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/985fcfc6698aeb0ea8a840ec0aa2391a… --- View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/985fcfc6698aeb0ea8a840ec0aa2391a… You're receiving this email because of your account on gitlab.com.

1 0

[Git][ultreiaio/ird-t3][develop] Sélection par défaut du type d'échantillons (closes #296)
by Tony CHEMIT 19 Mar '18

19 Mar '18

Tony CHEMIT pushed to branch develop at ultreiaio / ird-t3 Commits: f63dab1e by Tony CHEMIT at 2018-03-19T22:55:00Z Sélection par défaut du type d'échantillons (closes #296) - - - - - 1 changed file: - t3-web/src/main/java/fr/ird/t3/web/actions/data/level1/ManageLevel1ConfigurationAction.java Changes: ===================================== t3-web/src/main/java/fr/ird/t3/web/actions/data/level1/ManageLevel1ConfigurationAction.java ===================================== --- a/t3-web/src/main/java/fr/ird/t3/web/actions/data/level1/ManageLevel1ConfigurationAction.java +++ b/t3-web/src/main/java/fr/ird/t3/web/actions/data/level1/ManageLevel1ConfigurationAction.java @@ -129,7 +129,7 @@ public class ManageLevel1ConfigurationAction extends AbstractConfigureAction<Lev List<String> ids; // use default sample qualities - ids = T3EntityHelper.selectIdsByCodes(conf.getSampleQualities(), 1, 2, 3, 9); + ids = T3EntityHelper.selectIdsByCodes(conf.getSampleQualities(), 1); conf.setSampleQualityIds(ids); // use default sample types View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/f63dab1e6ee9f83f3c75770c722fcc55… --- View it on GitLab: https://gitlab.com/ultreiaio/ird-t3/commit/f63dab1e6ee9f83f3c75770c722fcc55… You're receiving this email because of your account on gitlab.com.

1 0