December 2010 - Wikitty-commits - mailman.cloud.codelutin.com

r634 - trunk
by jcouteau＠users.nuiton.org 20 Dec '10

20 Dec '10

Author: jcouteau Date: 2010-12-20 16:57:47 +0100 (Mon, 20 Dec 2010) New Revision: 634 Url: http://nuiton.org/repositories/revision/wikitty/634 Log: Remove useless dependencies from jcs Update commons-cli to 1.2 Modified: trunk/pom.xml Modified: trunk/pom.xml =================================================================== --- trunk/pom.xml 2010-12-20 14:55:16 UTC (rev 633) +++ trunk/pom.xml 2010-12-20 15:57:47 UTC (rev 634) @@ -42,6 +42,41 @@ <groupId>jcs</groupId> <artifactId>jcs</artifactId> <version>1.3</version> + <exclusions> + <exclusion> + <groupId>mysql</groupId> + <artifactId>mysql-connector-java</artifactId> + </exclusion> + <exclusion> + <groupId>xerces</groupId> + <artifactId>xercesImpl</artifactId> + </exclusion> + <exclusion> + <groupId>tomcat</groupId> + <artifactId>tomcat-util</artifactId> + </exclusion> + <exclusion> + <groupId>berkeleydb</groupId> + <artifactId>berkeleydb</artifactId> + </exclusion> + <exclusion> + <groupId>xmlrpc</groupId> + <artifactId>xmlrpc</artifactId> + </exclusion> + <exclusion> + <groupId>xml-apis</groupId> + <artifactId>xml-apis</artifactId> + </exclusion> + <exclusion> + <groupId>velocity</groupId> + <artifactId>velocity</artifactId> + </exclusion> + <exclusion> + <groupId>hsqldb</groupId> + <artifactId>hsqldb</artifactId> + </exclusion> + </exclusions> + </dependency>  @@ -276,7 +311,7 @@ <dependency> <groupId>commons-cli</groupId> <artifactId>commons-cli</artifactId> - <version>1.1</version> + <version>1.2</version> <scope>test</scope> </dependency>

1 0

r633 - in trunk: wikitty-dto/src/main/java/org/nuiton/wikitty/dto wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr
by jcouteau＠users.nuiton.org 20 Dec '10

20 Dec '10

Author: jcouteau Date: 2010-12-20 15:55:16 +0100 (Mon, 20 Dec 2010) New Revision: 633 Url: http://nuiton.org/repositories/revision/wikitty/633 Log: Ignore failing test Prevent NPE in DTO Modified: trunk/wikitty-dto/src/main/java/org/nuiton/wikitty/dto/DTOHelper.java trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/SolrSearchTest.java Modified: trunk/wikitty-dto/src/main/java/org/nuiton/wikitty/dto/DTOHelper.java =================================================================== --- trunk/wikitty-dto/src/main/java/org/nuiton/wikitty/dto/DTOHelper.java 2010-12-20 13:03:45 UTC (rev 632) +++ trunk/wikitty-dto/src/main/java/org/nuiton/wikitty/dto/DTOHelper.java 2010-12-20 14:55:16 UTC (rev 633) @@ -39,16 +39,19 @@ static Log log = LogFactory.getLog(DTOHelper.class); public static <E extends BusinessEntity> E toDto(E w) { - Class clazz = w.getClass(); - String dtoClassName = clazz.getName().replace("Impl", "DTO").replace("CopyOnWrite", "DTO"); E result = null; - try { - String wikittyId = w.getWikittyId(); - result = (E)ObjectUtil.newInstance(dtoClassName+"("+wikittyId+")"); - result.copyFrom(w); - }catch (Exception eee){ - log.error("Could not transform Wikitty to DTO", eee); + if (w != null){ + Class clazz = w.getClass(); + String dtoClassName = clazz.getName().replace("Impl", "DTO").replace("CopyOnWrite", "DTO"); + try { + String wikittyId = w.getWikittyId(); + result = (E)ObjectUtil.newInstance(dtoClassName+"("+wikittyId+")"); + + result.copyFrom(w); + }catch (Exception eee){ + log.error("Could not transform Wikitty to DTO", eee); + } } return result; } Modified: trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/SolrSearchTest.java =================================================================== --- trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/SolrSearchTest.java 2010-12-20 13:03:45 UTC (rev 632) +++ trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/SolrSearchTest.java 2010-12-20 14:55:16 UTC (rev 633) @@ -407,7 +407,7 @@ } // on ignore ce test car le like toLowercase devrait disparaitre et pour l'instant il ne marche pas :( -// @Ignore + @Ignore @Test public void testLikeSearch() throws Exception { Criteria criteria = Search.query().like("Category.name", "hArDwArE", Like.SearchAs.ToLowerCase).criteria();

1 0

r632 - in trunk: wikitty-api/src/main/java/org/nuiton/wikitty wikitty-solr-impl/src/main/java/org/nuiton/wikitty/storage/solr wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr
by bpoussin＠users.nuiton.org 20 Dec '10

20 Dec '10

Author: bpoussin Date: 2010-12-20 14:03:45 +0100 (Mon, 20 Dec 2010) New Revision: 632 Url: http://nuiton.org/repositories/revision/wikitty/632 Log: - WikittySearchEngineSolrIndexInTreeNode is completely implemented - add javadoc on WikittyService Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyService.java trunk/wikitty-solr-impl/src/main/java/org/nuiton/wikitty/storage/solr/WikittySearchEngineSolr.java trunk/wikitty-solr-impl/src/main/java/org/nuiton/wikitty/storage/solr/WikittySearchEngineSolrIndexInTreeNode.java trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/SolrSearchTest.java trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/TreeTest.java trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/WikittyServiceSolr.java Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java 2010-12-20 12:39:01 UTC (rev 631) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java 2010-12-20 13:03:45 UTC (rev 632) @@ -119,6 +119,7 @@ } /** + * TODO poussin 20101219 extract this class from WikittyConfig * Configuration option for all modules */ public static enum WikittyOption implements OptionDef { Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyService.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyService.java 2010-12-20 12:39:01 UTC (rev 631) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyService.java 2010-12-20 13:03:45 UTC (rev 632) @@ -338,21 +338,29 @@ /** * Retrieve wikitty node with count. Wikitty reference by wikittyId MUST - * include the 'Node' extension. - * + * include the 'WikittyTreeNode' extension. + * + * Count is number of attachment in subtree. If filter is not null only + * attachments that satisfy filter are counted + * * @param securityToken security token * @param wikittyId + * @param filter * @return */ public Map.Entry<WikittyTreeNode, Integer> restoreNode( String securityToken, String wikittyId, Criteria filter); /** - * Retrieve all wikitties children with count (no recursively) of an other one - * Wikitty reference by wikittyId MUST include the 'Node' extension + * Retrieve all wikitties children (no recursively) with count of an other one + * Wikitty reference by wikittyId MUST include the 'WikittyTreeNode' extension * + * Count is number of attachment in subtree (recursively). If filter is not + * null only attachments that satisfy filter are counted + * * @param securityToken security token * @param wikittyId + * @param filter * @return */ public Map<WikittyTreeNode, Integer> restoreChildren( Modified: trunk/wikitty-solr-impl/src/main/java/org/nuiton/wikitty/storage/solr/WikittySearchEngineSolr.java =================================================================== --- trunk/wikitty-solr-impl/src/main/java/org/nuiton/wikitty/storage/solr/WikittySearchEngineSolr.java 2010-12-20 12:39:01 UTC (rev 631) +++ trunk/wikitty-solr-impl/src/main/java/org/nuiton/wikitty/storage/solr/WikittySearchEngineSolr.java 2010-12-20 13:03:45 UTC (rev 632) @@ -27,7 +27,6 @@ import static org.nuiton.wikitty.storage.solr.WikittySolrConstant.*; -import java.io.IOException; import java.util.ArrayList; import java.util.Collection; import java.util.HashMap; @@ -57,21 +56,12 @@ import org.nuiton.wikitty.entities.WikittyTreeNode; import org.nuiton.wikitty.entities.Wikitty; import org.nuiton.wikitty.WikittyException; -import org.nuiton.wikitty.entities.WikittyExtension; import org.nuiton.wikitty.storage.WikittyExtensionStorage; import org.nuiton.wikitty.storage.WikittySearchEngine; import org.nuiton.wikitty.services.WikittyTransaction; -import org.nuiton.wikitty.search.operators.Element; import org.nuiton.wikitty.search.Search; -import com.arjuna.ats.arjuna.coordinator.BasicAction; -import com.arjuna.ats.arjuna.coordinator.OnePhaseResource; -import com.arjuna.ats.arjuna.coordinator.TwoPhaseOutcome; -import com.arjuna.ats.arjuna.state.InputObjectState; -import com.arjuna.ats.arjuna.state.OutputObjectState; -import com.arjuna.ats.internal.arjuna.abstractrecords.LastResourceRecord; import java.io.File; -import java.util.regex.Pattern; import org.nuiton.util.ApplicationConfig; import org.nuiton.wikitty.WikittyConfig; import org.nuiton.wikitty.WikittyUtil; Modified: trunk/wikitty-solr-impl/src/main/java/org/nuiton/wikitty/storage/solr/WikittySearchEngineSolrIndexInTreeNode.java =================================================================== --- trunk/wikitty-solr-impl/src/main/java/org/nuiton/wikitty/storage/solr/WikittySearchEngineSolrIndexInTreeNode.java 2010-12-20 12:39:01 UTC (rev 631) +++ trunk/wikitty-solr-impl/src/main/java/org/nuiton/wikitty/storage/solr/WikittySearchEngineSolrIndexInTreeNode.java 2010-12-20 13:03:45 UTC (rev 632) @@ -61,6 +61,7 @@ import org.nuiton.wikitty.services.WikittyTransaction; import java.io.File; +import java.util.Collections; import java.util.LinkedHashSet; import java.util.LinkedList; import org.nuiton.util.ApplicationConfig; @@ -477,39 +478,65 @@ } } + /** + * Cette implantation n'est pas vraiment optimale en place memoire si + * le 'filter' retourne enormement de resultat ou qu'il y a beaucoup + * d'attachment dans l'arbre. + * + * On a en memoire l'ensemble des ids des attachments du sous arbre et + * l'ensemble des ids du resultat du filter. + * + * @param transaction + * @param w + * @param filter + * @return + */ @Override public Integer findNodeCount(WikittyTransaction transaction, Wikitty w, Criteria filter) { try { int result = 0; String id = w.getId(); - if (filter == null) { - // we used hashSet to count only once same attachments in many node - Collection allAttachments = new HashSet(); + // we used hashSet to count only once same attachments in many node + Collection allAttachments = new HashSet(); - // on recherche tout les fils du noeud - SolrQuery query = new SolrQuery(SOLR_QUERY_PARSER + TREENODE_PARENTS + ":" + id); - QueryResponse response = solrServer.query(query); - SolrDocumentList updateDocs = response.getResults(); - for (Iterator<SolrDocument> iterator = updateDocs.iterator(); iterator.hasNext();) { - SolrDocument solrDocument = iterator.next(); - Collection attachments = solrDocument.getFieldValues( - WikittyTreeNode.FQ_FIELD_WIKITTYTREENODE_ATTACHMENT - + SUFFIX_WIKITTY); - if (attachments != null) { - allAttachments.addAll(attachments); - } + // on recherche tout les fils du noeud + SolrQuery query = new SolrQuery(SOLR_QUERY_PARSER + TREENODE_PARENTS + ":" + id); + QueryResponse response = solrServer.query(query); + SolrDocumentList updateDocs = response.getResults(); + for (Iterator<SolrDocument> iterator = updateDocs.iterator(); iterator.hasNext();) { + SolrDocument solrDocument = iterator.next(); + Collection attachments = solrDocument.getFieldValues( + WikittyTreeNode.FQ_FIELD_WIKITTYTREENODE_ATTACHMENT + + SUFFIX_WIKITTY); + if (attachments != null) { + allAttachments.addAll(attachments); } - result = allAttachments.size(); - } else { - throw new UnsupportedOperationException("Not yet implemented with filter!"); } + if (filter != null) { + PagedResult<String> criteriaResult = findAllByCriteria(transaction, filter); + allAttachments.retainAll(criteriaResult.getAll()); + } + result = allAttachments.size(); return result; } catch (SolrServerException eee) { throw new WikittyException("Can't find node count", eee); } } + /** + * Cette implantation n'est pas vraiment optimale en place memoire si + * le 'filter' retourne enormement de resultat ou qu'il y a beaucoup + * d'attachment dans l'arbre. + * + * On a en memoire l'ensemble des ids des attachments du sous arbre et + * l'ensemble des ids du resultat du filter. + * + * @param transaction + * @param w + * @param filter + * @return + */ @Override public Map<String, Integer> findAllChildrenCount( WikittyTransaction transaction, Wikitty w, Criteria filter) { @@ -518,56 +545,65 @@ Map<String, Integer> result = new HashMap<String, Integer>(); String id = w.getId(); - if (filter == null) { // key: id node; value: attachment count for this node (with sub*node) - Map<String, Collection> allAttachments = new HashMap<String, Collection>(); - - // les id des fils direct du wikitty passe en parametre - Set<String> child = new HashSet<String>(); + Map<String, Collection> allAttachments = new HashMap<String, Collection>(); - // on recherche tout les fils et sous fils, lui compris - SolrQuery query = new SolrQuery(SOLR_QUERY_PARSER + TREENODE_PARENTS + ":" + id); - QueryResponse response = solrServer.query(query); - SolrDocumentList updateDocs = response.getResults(); + // les id des fils direct du wikitty passe en parametre + Set<String> child = new HashSet<String>(); - for (Iterator<SolrDocument> iterator = updateDocs.iterator(); iterator.hasNext();) { - SolrDocument solrDocument = iterator.next(); - String childId = (String)solrDocument.getFieldValue(SOLR_ID); - String parentId = (String)solrDocument.getFieldValue( - WikittyTreeNode.FQ_FIELD_WIKITTYTREENODE_PARENT + SUFFIX_WIKITTY); - Collection attachments = solrDocument.getFieldValues( - WikittyTreeNode.FQ_FIELD_WIKITTYTREENODE_ATTACHMENT + SUFFIX_WIKITTY); - Collection parents = solrDocument.getFieldValues( - TREENODE_PARENTS); + // on recherche tout les fils et sous fils, lui compris + SolrQuery query = new SolrQuery(SOLR_QUERY_PARSER + TREENODE_PARENTS + ":" + id); + QueryResponse response = solrServer.query(query); + SolrDocumentList updateDocs = response.getResults(); - if (id.equals(parentId)) { - // c'est un fils direct on l'ajoute a la liste des fils - child.add(childId); - } + for (Iterator<SolrDocument> iterator = updateDocs.iterator(); iterator.hasNext();) { + SolrDocument solrDocument = iterator.next(); + String childId = (String) solrDocument.getFieldValue(SOLR_ID); + String parentId = (String) solrDocument.getFieldValue( + WikittyTreeNode.FQ_FIELD_WIKITTYTREENODE_PARENT + SUFFIX_WIKITTY); + Collection attachments = solrDocument.getFieldValues( + WikittyTreeNode.FQ_FIELD_WIKITTYTREENODE_ATTACHMENT + SUFFIX_WIKITTY); + Collection parents = solrDocument.getFieldValues( + TREENODE_PARENTS); - for (Object p : parents) { - String parent = (String) p; - Collection col = allAttachments.get(parent); - if (col == null) { - // we used hashSet to count only once same attachments in many node - col = new HashSet(); - allAttachments.put(parent, col); - } - if (attachments != null) { - col.addAll(attachments); - } - } + if (id.equals(parentId)) { + // c'est un fils direct on l'ajoute a la liste des fils + child.add(childId); } - for (Map.Entry<String, Collection> e : allAttachments.entrySet()) { - // on ne garde que les fils directs - if (child.contains(e.getKey())) { - result.put(e.getKey(), e.getValue().size()); + for (Object p : parents) { + String parent = (String) p; + Collection col = allAttachments.get(parent); + if (col == null) { + // we used hashSet to count only once same attachments in many node + col = new HashSet(); + allAttachments.put(parent, col); } + if (attachments != null) { + col.addAll(attachments); + } } - } else { - throw new UnsupportedOperationException("Not yet implemented with filter!"); } + + // creation des id repondants au filtre si besoin + Collection<String> filteredId = Collections.EMPTY_LIST; + if (filter != null) { + PagedResult<String> criteriaResult = findAllByCriteria(transaction, filter); + filteredId = criteriaResult.getAll(); + } + + // Construction du resultat + for (Map.Entry<String, Collection> e : allAttachments.entrySet()) { + // on ne garde que les fils directs + if (child.contains(e.getKey())) { + Collection attachments = e.getValue(); + // on ne garde dans le comptage des attachments que les id + // qui reponde aussi au filter + attachments.retainAll(filteredId); + result.put(e.getKey(), attachments.size()); + } + } + return result; } catch (SolrServerException eee) { throw new WikittyException("Can't find node count", eee); Modified: trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/SolrSearchTest.java =================================================================== --- trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/SolrSearchTest.java 2010-12-20 12:39:01 UTC (rev 631) +++ trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/SolrSearchTest.java 2010-12-20 13:03:45 UTC (rev 632) @@ -407,7 +407,7 @@ } // on ignore ce test car le like toLowercase devrait disparaitre et pour l'instant il ne marche pas :( - @Ignore +// @Ignore @Test public void testLikeSearch() throws Exception { Criteria criteria = Search.query().like("Category.name", "hArDwArE", Like.SearchAs.ToLowerCase).criteria(); Modified: trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/TreeTest.java =================================================================== --- trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/TreeTest.java 2010-12-20 12:39:01 UTC (rev 631) +++ trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/TreeTest.java 2010-12-20 13:03:45 UTC (rev 632) @@ -270,6 +270,7 @@ Criteria filter = Search.query().eq("test.name", "value 3").criteria(); Map<WikittyTreeNode, Integer> children = ws.restoreChildren(null, node1Id, filter); + System.out.println(children); assertEquals(3, children.size()); } @@ -289,6 +290,7 @@ Criteria filter = Search.query().eq("test.name", "value 3").criteria(); Entry<WikittyTreeNode, Integer> count = ws.restoreNode(null, node11Id, filter); + System.out.println(count); assertEquals(1, count.getValue().intValue()); } Modified: trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/WikittyServiceSolr.java =================================================================== --- trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/WikittyServiceSolr.java 2010-12-20 12:39:01 UTC (rev 631) +++ trunk/wikitty-solr-impl/src/test/java/org/nuiton/wikitty/storage/solr/WikittyServiceSolr.java 2010-12-20 13:03:45 UTC (rev 632) @@ -27,8 +27,6 @@ import org.nuiton.util.ApplicationConfig; import org.nuiton.wikitty.services.WikittyServiceStorage; -import org.nuiton.wikitty.storage.solr.WikittySearchEngineSolr; -import org.nuiton.wikitty.storage.solr.WikittySearchEngineSolrIndexInTreeNode; import org.nuiton.wikitty.storage.WikittyExtensionStorageInMemory; import org.nuiton.wikitty.storage.WikittyStorageInMemory;

1 0

r631 - trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services
by bpoussin＠users.nuiton.org 20 Dec '10

20 Dec '10

Author: bpoussin Date: 2010-12-20 13:39:01 +0100 (Mon, 20 Dec 2010) New Revision: 631 Url: http://nuiton.org/repositories/revision/wikitty/631 Log: suppression de message de debug en sop Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java 2010-12-19 13:19:10 UTC (rev 630) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java 2010-12-20 12:39:01 UTC (rev 631) @@ -132,14 +132,13 @@ protected void statAdd(int asked, int missed) { this.asked += asked; this.missed += missed; -// if(log.isDebugEnabled()) { + if(log.isDebugEnabled()) { String total = StringUtil.convertMemory(Runtime.getRuntime().totalMemory()); String free = StringUtil.convertMemory(Runtime.getRuntime().freeMemory()); String msg = String.format("cache stat (missed/asked): %s/%s (memory %s/%s [total/free])", this.missed, this.asked, total, free); log.debug(msg); - System.out.println(msg); -// } + } } /**

1 0

r630 - trunk/wikitty-api/src/main/java/org/nuiton/wikitty
by bpoussin＠users.nuiton.org 19 Dec '10

19 Dec '10

Author: bpoussin Date: 2010-12-19 14:19:10 +0100 (Sun, 19 Dec 2010) New Revision: 630 Url: http://nuiton.org/repositories/revision/wikitty/630 Log: fix miss spelling option name Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java 2010-12-19 12:44:26 UTC (rev 629) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java 2010-12-19 13:19:10 UTC (rev 630) @@ -235,7 +235,7 @@ "", String.class, false, false), WIKITTY_CACHE_SIMPLE_REFERENCE_TYPE( - "wikitty.service.cache.simple", + "wikitty.service.cache.simple.reference.type", _("Indique le type de reference a utiliser pour les extensions" + " non prioritaires dans le WikittyCacheSimple (soft ou hard)"), "soft", String.class, false, false),

1 0

r629 - in trunk/wikitty-api/src/main/java/org/nuiton/wikitty: . services
by bpoussin＠users.nuiton.org 19 Dec '10

19 Dec '10

Author: bpoussin Date: 2010-12-19 13:44:26 +0100 (Sun, 19 Dec 2010) New Revision: 629 Url: http://nuiton.org/repositories/revision/wikitty/629 Log: enhance cache feature - we can specify priority extension, less removed from cache that other wikitty) - support cache for extension Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCache.java trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheJCS.java trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheSimple.java trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java 2010-12-19 05:17:12 UTC (rev 628) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java 2010-12-19 12:44:26 UTC (rev 629) @@ -229,6 +229,17 @@ _("Indique le composant a utiliser pour le cache"), WikittyCacheJCS.class.getName(), String.class, false, false), + WIKITTY_CACHE_PRIORITY_EXTENSIONS( + "wikitty.service.cache.priority.extensions", + _("Indique la liste des extensions qui doivent etre conservee prioritairement"), + "", String.class, false, false), + + WIKITTY_CACHE_SIMPLE_REFERENCE_TYPE( + "wikitty.service.cache.simple", + _("Indique le type de reference a utiliser pour les extensions" + + " non prioritaires dans le WikittyCacheSimple (soft ou hard)"), + "soft", String.class, false, false), + WIKITTY_WIKITTYSERVICEACCESSSTAT_COMPONENTS( "wikitty.WikittyServiceAccessStat.components", _("Indique le composant a utiliser pour le stockage des acces"), Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCache.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCache.java 2010-12-19 05:17:12 UTC (rev 628) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCache.java 2010-12-19 12:44:26 UTC (rev 629) @@ -1,9 +1,14 @@ package org.nuiton.wikitty.services; import org.nuiton.wikitty.entities.Wikitty; +import org.nuiton.wikitty.entities.WikittyExtension; /** + * Interface commune a toutes les implantations du cache. Les implantations + * doivent faire le moins de travail possible, donc s'il y a des choses communes + * il faut les coder dans {@link WikittyServiceCached} * + * * @author poussin * @version $Revision$ * @@ -28,7 +33,7 @@ Wikitty getWikitty(String id); /** - * Only realy put wikitty in cache, if not in cache or version is newer than + * put wikitty in cache * one in cache * @param e */ @@ -41,4 +46,32 @@ */ void removeWikitty(String id); + public boolean existsExtension(String id); + + /** + * Return extension object if is in the cache, null otherwize. + * + * @param id + * @return extension object or null + */ + public WikittyExtension getExtension(String id); + + /** + * put extension in cache + * @param e + */ + public void putExtension(WikittyExtension e); + + /** + * Remove extension from cache. + * + * @param id wikitty id to remove + */ + public void removeExtension(String id); + + /** + * Clear all cache. + */ + public void clearExtension(); + } Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheJCS.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheJCS.java 2010-12-19 05:17:12 UTC (rev 628) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheJCS.java 2010-12-19 12:44:26 UTC (rev 629) @@ -25,16 +25,21 @@ package org.nuiton.wikitty.services; +import java.util.Collections; +import java.util.HashSet; +import java.util.List; +import java.util.Set; import org.nuiton.wikitty.entities.Wikitty; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; -import org.apache.jcs.JCS; import org.apache.jcs.access.GroupCacheAccess; import org.apache.jcs.access.exception.CacheException; import org.apache.jcs.engine.control.CompositeCacheManager; import org.nuiton.util.ApplicationConfig; +import org.nuiton.wikitty.WikittyConfig; import org.nuiton.wikitty.WikittyException; +import org.nuiton.wikitty.entities.WikittyExtension; /** * Implantation du cache base sur le projet apache JCS @@ -48,19 +53,40 @@ public class WikittyCacheJCS implements WikittyCache { static private Log log = LogFactory.getLog(WikittyCacheJCS.class); - + + protected Set<String> priorityExtensions = new HashSet<String>(); + + /** + * cache ou sont stocke les objects qui doivent rester le plus possible + * en memoire + */ + protected GroupCacheAccess priorityCache; + /** + * cache pour les autres wikitties + */ protected GroupCacheAccess wikittyCache; /** + * cache pour les extensions + */ + protected GroupCacheAccess extensionCache; + + /** * * @param config not used currently but necessary in futur to configure the cache * Create a soft cache. */ public WikittyCacheJCS(ApplicationConfig config) { try { + List<String> priorityExtensions = config.getOptionAsList( + WikittyConfig.WikittyOption.WIKITTY_CACHE_PRIORITY_EXTENSIONS.getKey()).getOption(); + this.priorityExtensions.addAll(priorityExtensions); + CompositeCacheManager cacheMgr = CompositeCacheManager.getUnconfiguredInstance(); cacheMgr.configure(config.getFlatOptions()); + priorityCache = new GroupCacheAccess(cacheMgr.getCache("priority")); wikittyCache = new GroupCacheAccess(cacheMgr.getCache("wikitty")); + extensionCache = new GroupCacheAccess(cacheMgr.getCache("extension")); // JCS ne permet pas de passer un Properties mais seulement un nom de fichier :( // JCS.setConfigFilename(config.getConfigFileName()); @@ -73,8 +99,13 @@ @Override public boolean existsWikitty(String id) { - Object o = wikittyCache.get(id); + Object o = priorityCache.get(id); boolean result = (o != null); + if (!result) { + o = wikittyCache.get(id); + result = (o != null); + } + return result; } @@ -86,7 +117,10 @@ */ @Override public Wikitty getWikitty(String id) { - Wikitty result = (Wikitty)wikittyCache.get(id); + Wikitty result = (Wikitty)priorityCache.get(id); + if (result == null) { + result = (Wikitty) wikittyCache.get(id); + } return result; } @@ -99,7 +133,13 @@ public void putWikitty(Wikitty e) { if (e != null) { try { - wikittyCache.put(e.getId(), e); + if (Collections.disjoint(priorityExtensions,e.getExtensionNames())) { + // le wikitty ne contient pas d'extension prioritaire + // on le met dans le cache commun + wikittyCache.put(e.getId(), e); + } else { + priorityCache.put(e.getId(), e); + } } catch (CacheException eee) { log.error(String.format("Can't put wikitty %s in cache", e), eee); } @@ -114,6 +154,7 @@ @Override public void removeWikitty(String id) { try { + priorityCache.remove(id); wikittyCache.remove(id); } catch (CacheException eee) { log.error(String.format("Can't remove wikitty %s in cache", id), eee); @@ -126,10 +167,71 @@ @Override public void clearWikitty() { try { + priorityCache.clear(); wikittyCache.clear(); } catch (CacheException eee) { log.error(String.format("Can't clear wikitty cache"), eee); } } + @Override + public boolean existsExtension(String id) { + Object o = extensionCache.get(id); + boolean result = (o != null); + return result; + } + + /** + * Return wikitty object if is in the cache, null otherwize. + * + * @param id + * @return wikitty object or null + */ + @Override + public WikittyExtension getExtension(String id) { + WikittyExtension result = (WikittyExtension)extensionCache.get(id); + return result; + } + + /** + * Only realy put wikitty in cache, if not in cache or version is newer than + * one in cache + * @param e + */ + @Override + public void putExtension(WikittyExtension e) { + if (e != null) { + try { + extensionCache.put(e.getId(), e); + } catch (CacheException eee) { + log.error(String.format("Can't put extension %s in cache", e), eee); + } + } + } + + /** + * Remove wikitty from cache. + * + * @param id wikitty id to remove + */ + @Override + public void removeExtension(String id) { + try { + extensionCache.remove(id); + } catch (CacheException eee) { + log.error(String.format("Can't remove extensions %s in cache", id), eee); + } + } + + /** + * Clear all cache. + */ + @Override + public void clearExtension() { + try { + extensionCache.clear(); + } catch (CacheException eee) { + log.error(String.format("Can't clear extension cache"), eee); + } + } } Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheSimple.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheSimple.java 2010-12-19 05:17:12 UTC (rev 628) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheSimple.java 2010-12-19 12:44:26 UTC (rev 629) @@ -25,14 +25,20 @@ package org.nuiton.wikitty.services; +import java.util.Collections; import org.nuiton.wikitty.entities.Wikitty; import java.util.HashMap; +import java.util.HashSet; +import java.util.List; import java.util.Map; +import java.util.Set; import org.apache.commons.collections.map.ReferenceMap; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.nuiton.util.ApplicationConfig; +import org.nuiton.wikitty.WikittyConfig; +import org.nuiton.wikitty.entities.WikittyExtension; /** * Cette classe sert a introduire du cache dans wikitty. Elle sert a centraliser @@ -49,36 +55,58 @@ static private Log log = LogFactory.getLog(WikittyCacheSimple.class); - /** keys are wikitty ids */ + protected Set<String> priorityExtensions = new HashSet<String>(); + + /** + * cache ou sont stocke les objects qui doivent rester le plus possible + * en memoire. + * ATTENTION: ce ne sont que des references HARD, donc il ne faut pas + * qu'il y ait trop d'objet prioritaire, sinon l'application risque de + * manquer de memoire + * keys are wikitty ids + */ + protected Map<String, Wikitty> priorityCache = new HashMap<String, Wikitty>(); + + /** + * cache pour les autres wikitties + * keys are wikitty ids + */ protected Map<String, Wikitty> wikittyCache; + protected Map<String, WikittyExtension> extensionCache; + /** * * @param config not used currently but necessary in futur to configure the cache * Create a soft cache. */ public WikittyCacheSimple(ApplicationConfig config) { - this(true); - } + List<String> priorityExtensions = config.getOptionAsList( + WikittyConfig.WikittyOption.WIKITTY_CACHE_PRIORITY_EXTENSIONS.getKey()).getOption(); + this.priorityExtensions.addAll(priorityExtensions); - /** - * Init cache with specific reference type. - * - * @param soft si vrai utilise un cache avec des reference Soft, sinon - * utilise des references static (utile pour les transactions) - */ - public WikittyCacheSimple(boolean soft) { - if (soft) { + String refType = config.getOption( + WikittyConfig.WikittyOption.WIKITTY_CACHE_SIMPLE_REFERENCE_TYPE.getKey()); + if ("soft".equalsIgnoreCase(refType)) { wikittyCache = new ReferenceMap(ReferenceMap.HARD, ReferenceMap.SOFT); + extensionCache = new ReferenceMap(ReferenceMap.HARD, ReferenceMap.SOFT); } else { + // HARD reference use normal map wikittyCache = new HashMap<String, Wikitty>(); - } + extensionCache = new HashMap<String, WikittyExtension>(); + } + } @Override public boolean existsWikitty(String id) { - Object o = wikittyCache.get(id); + Object o = priorityCache.get(id); boolean result = (o != null); + if (!result) { + o = wikittyCache.get(id); + result = (o != null); + } + return result; } @@ -90,7 +118,10 @@ */ @Override public Wikitty getWikitty(String id) { - Wikitty result = wikittyCache.get(id); + Wikitty result = (Wikitty)priorityCache.get(id); + if (result == null) { + result = (Wikitty) wikittyCache.get(id); + } return result; } @@ -102,7 +133,13 @@ @Override public void putWikitty(Wikitty e) { if (e != null) { - wikittyCache.put(e.getId(), e); + if (Collections.disjoint(priorityExtensions, e.getExtensionNames())) { + // le wikitty ne contient pas d'extension prioritaire + // on le met dans le cache commun + wikittyCache.put(e.getId(), e); + } else { + priorityCache.put(e.getId(), e); + } } } @@ -113,6 +150,7 @@ */ @Override public void removeWikitty(String id) { + priorityCache.remove(id); wikittyCache.remove(id); } @@ -121,7 +159,38 @@ */ @Override public void clearWikitty() { + priorityCache.clear(); wikittyCache.clear(); } + @Override + public boolean existsExtension(String id) { + Object o = extensionCache.get(id); + boolean result = (o != null); + return result; + } + + @Override + public WikittyExtension getExtension(String id) { + WikittyExtension result = (WikittyExtension)extensionCache.get(id); + return result; + } + + @Override + public void putExtension(WikittyExtension e) { + if (e != null) { + extensionCache.put(e.getId(), e); + } + } + + @Override + public void removeExtension(String id) { + extensionCache.remove(id); + } + + @Override + public void clearExtension() { + extensionCache.clear(); + } + } Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java 2010-12-19 05:17:12 UTC (rev 628) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java 2010-12-19 12:44:26 UTC (rev 629) @@ -142,6 +142,23 @@ // } } + /** + * Retourne le nombre de solicitation du cache + * @return + */ + public long getAsked() { + return asked; + } + + /** + * Retourne le nombre de fois que l'element dans le cache demande n'y etait + * pas + * @return + */ + public long getMissed() { + return missed; + } + /** wrap the wikitty or copy it according to allwaysRestoreCopies value */ protected Wikitty wrapWikitty(Wikitty wikitty) { Wikitty result = null; @@ -176,21 +193,25 @@ // son access // - protected void clearCache() { + protected void cacheClearWikitty() { cache.clearWikitty(); } + protected void cacheClearExtension() { + cache.clearExtension(); + } + /** * Only WikittyImpl can be put in real cache implementation. If argument * is WikittyCopyOnWrite, we must take internal wikitty to put in cache */ - protected void putInCache(Wikitty w) { + protected void cachePutWikitty(Wikitty w) { if (w != null) { if (w instanceof WikittyCopyOnWrite) { w = ((WikittyCopyOnWrite) w).getTarget(); } - Wikitty old = getFromCache(w.getId()); + Wikitty old = cache.getWikitty(w.getId()); if (old == null || WikittyUtil.versionGreaterThan(w.getVersion(), old.getVersion())) { cache.putWikitty(w); @@ -208,30 +229,72 @@ } } } - - protected void putInCache(Collection<Wikitty> wikitties) { + + /** + * Only WikittyImpl can be put in real cache implementation. If argument + * is WikittyCopyOnWrite, we must take internal wikitty to put in cache + */ + protected void cachePutExtension(WikittyExtension w) { + if (w != null) { + if (!cache.existsExtension(w.getId())) { + cache.putExtension(w); + + if (log.isTraceEnabled()) { + log.trace(String.format( + "Replace cached wikitty extension '%s'", w.getId())); + } + } else { + if (log.isTraceEnabled()) { + log.trace(String.format( + "Ignoring put wikitty extension for '%s'",w.getId())); + } + } + } + } + + protected void cachePutWikitty(Collection<Wikitty> wikitties) { for (Wikitty w : wikitties) { - putInCache(w); + cachePutWikitty(w); } } - protected void removeInCache(String id) { + protected void cachePutExtension(Collection<WikittyExtension> wikitties) { + for (WikittyExtension w : wikitties) { + cachePutExtension(w); + } + } + protected void cacheRemoveWikitty(String id) { cache.removeWikitty(id); } - protected void removeInCache(Collection<String> ids) { + protected void cacheRemoveExtension(String id) { + cache.removeWikitty(id); + } + + protected void cacheRemoveWikitty(Collection<String> ids) { for (String id : ids) { - removeInCache(id); + cacheRemoveWikitty(id); } } - protected Wikitty getFromCache(String id) { + protected void cacheRemoveExtension(Collection<String> ids) { + for (String id : ids) { + cacheRemoveWikitty(id); + } + } + + protected Wikitty cacheGetWikitty(String id) { Wikitty result = cache.getWikitty(id); // all time wrap, inly WikittyImpl are in cache result = wrapWikitty(result); return result; } + protected WikittyExtension cacheGetExtensions(String id) { + WikittyExtension result = cache.getExtension(id); + return result; + } + // // surcharge des methodes du WikittyService // @@ -239,7 +302,7 @@ @Override public WikittyEvent clear(String securityToken) { WikittyEvent result = ws.clear(securityToken); - clearCache(); + processEvent(result); return result; } @@ -250,7 +313,7 @@ @Override public WikittyEvent delete(String securityToken, Collection<String> ids) { WikittyEvent result = ws.delete(securityToken, ids); - removeInCache(ids); + processEvent(result); return result; } @@ -291,7 +354,8 @@ */ @Override public List<String> getAllExtensionIds(String securityToken) { - // TODO poussin 20100412: perhaps use cache for extension ? + // NOTE poussin 20101219: si on veut utiliser le cache il faut une + // methode specifique return ws.getAllExtensionIds(securityToken); } @@ -322,7 +386,7 @@ LinkedHashMap<String, Wikitty> fromCache = new LinkedHashMap<String, Wikitty>(); for (String id : ids) { - Wikitty w = getFromCache(id); + Wikitty w = cacheGetWikitty(id); fromCache.put(id, w); // put all to maintains order if (w == null) { // if not found on cache, ask the server notInCache.add(id); @@ -332,7 +396,7 @@ // retrieve missing object List<Wikitty> missingInCache = ws.restore(securityToken, notInCache); - putInCache(missingInCache); + cachePutWikitty(missingInCache); for (Wikitty w : missingInCache) { // add missing object @@ -377,8 +441,16 @@ @Override public WikittyExtension restoreExtensionLastVersion( String securityToken, String name) { - // TODO poussin 20100412: perhaps use cache for extension ? - return ws.restoreExtensionLastVersion(securityToken, name); + // NOTE poussin 20100412: comment faire pour utiliser le cache ? on ne + // peut que stocker le resultat sinon il faudrait etre sur d'avoir toutes + // les extensions dans le cache. + // Sinon il faudrait mettre des methodes specifiques dans le cache pour + // cette methode, mise en cache lors du 1er appel, et restitution du + // meme resultat pour les appels suivants tant que pas de modification + // de l'extension (ou de clear/remove) + WikittyExtension result = ws.restoreExtensionLastVersion(securityToken, name); + cachePutExtension(result); + return result; } /** @@ -415,7 +487,9 @@ */ @Override public WikittyEvent deleteTree(String securityToken, String wikittyId) { - return ws.deleteTree(securityToken, wikittyId); + WikittyEvent result = ws.deleteTree(securityToken, wikittyId); + processEvent(result); + return result; } /** @@ -431,10 +505,7 @@ Collection<Wikitty> wikitties, boolean force) { WikittyEvent result = ws.store(securityToken, wikitties, force); - Map<String, Wikitty> wikittiesToCache = result.getWikitties(); - if (wikittiesToCache != null) { - putInCache(wikittiesToCache.values()); - } + processEvent(result); return result; } @@ -442,21 +513,24 @@ @Override public WikittyEvent storeExtension(String securityToken, Collection<WikittyExtension> exts) { - // TODO poussin 20101029: perhaps use cache for extension ? - return ws.storeExtension(securityToken, exts); + WikittyEvent result = ws.storeExtension(securityToken, exts); + processEvent(result); + return result; } @Override public WikittyEvent deleteExtension( String securityToken, Collection<String> extNames) { - // TODO poussin 20101115: perhaps use cache for extension ? - return ws.deleteExtension(securityToken, extNames); + WikittyEvent result = ws.deleteExtension(securityToken, extNames); + processEvent(result); + return result; } @Override public WikittyExtension restoreExtension(String securityToken, String id) { - // TODO poussin 20101029: perhaps use cache for extension ? - return ws.restoreExtension(securityToken, id); + WikittyExtension result = ws.restoreExtension(securityToken, id); + cachePutExtension(result); + return result; } @Override @@ -466,6 +540,14 @@ return ws.restoreVersion(securityToken, wikittyId, version); } + @Override + public WikittyEvent replay( + String securityToken, List<WikittyEvent> events, boolean force) { + WikittyEvent result = ws.replay(securityToken, events, force); + processEvent(result); + return result; + } + // // Just delegate method // @@ -511,12 +593,6 @@ } @Override - public WikittyEvent replay( - String securityToken, List<WikittyEvent> events, boolean force) { - return ws.replay(securityToken, events, force); - } - - @Override public boolean exists(String securityToken, String wikittyId) { return ws.exists(securityToken, wikittyId); } @@ -526,7 +602,30 @@ return ws.isDeleted(securityToken, wikittyId); } - + protected void processEvent(WikittyEvent e) { + // check clear must be the first, if event have clear and other type + // clear is all time play first + if (e.getType().contains( + WikittyEvent.WikittyEventType.CLEAR_WIKITTY) + || e.getType().contains( + WikittyEvent.WikittyEventType.CLEAR_EXTENSION)) { + cacheClearWikitty(); + cacheClearExtension(); + } else { + if (e.getType().contains(WikittyEvent.WikittyEventType.PUT_WIKITTY)) { + cachePutWikitty(e.getWikitties().values()); + } + if (e.getType().contains(WikittyEvent.WikittyEventType.REMOVE_WIKITTY)) { + cacheRemoveWikitty(e.getRemoveDate().keySet()); + } + if (e.getType().contains(WikittyEvent.WikittyEventType.PUT_EXTENSION)) { + cachePutExtension(e.getExtensions().values()); + } + if (e.getType().contains(WikittyEvent.WikittyEventType.REMOVE_EXTENSION)) { + cacheRemoveExtension(e.getDeletedExtensions()); + } + } + } /** * Classe permettant de recevoir les events distants et mettre a jour le cache */ @@ -543,7 +642,7 @@ */ @Override public void clearWikitty(WikittyEvent event) { - wsCached.clearCache(); + wsCached.processEvent(event); } /* @@ -551,8 +650,7 @@ */ @Override public void putWikitty(WikittyEvent event) { - Collection<Wikitty> wikitties = event.getWikitties().values(); - wsCached.putInCache(wikitties); + wsCached.processEvent(event); } /* @@ -560,10 +658,7 @@ */ @Override public void removeWikitty(WikittyEvent event) { - if (event.getRemoveDate() != null) { - Collection<String> ids = event.getRemoveDate().keySet(); - wsCached.removeInCache(ids); - } + wsCached.processEvent(event); } /* @@ -571,10 +666,12 @@ */ @Override public void putExtension(WikittyEvent event) { + wsCached.processEvent(event); } @Override public void removeExtension(WikittyEvent event) { + wsCached.processEvent(event); } /* @@ -582,7 +679,7 @@ */ @Override public void clearExtension(WikittyEvent event) { - wsCached.clearCache(); + wsCached.processEvent(event); } }

1 0

r628 - trunk/wikitty-api/src/main/java/org/nuiton/wikitty
by bpoussin＠users.nuiton.org 19 Dec '10

19 Dec '10

Author: bpoussin Date: 2010-12-19 06:17:12 +0100 (Sun, 19 Dec 2010) New Revision: 628 Url: http://nuiton.org/repositories/revision/wikitty/628 Log: - remove unused import Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java 2010-12-19 03:21:36 UTC (rev 627) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java 2010-12-19 05:17:12 UTC (rev 628) @@ -34,7 +34,6 @@ import org.apache.commons.logging.LogFactory; import org.nuiton.util.ApplicationConfig; import org.nuiton.util.ArgumentsParserException; -import org.nuiton.wikitty.services.WikittyCache; import org.nuiton.wikitty.services.WikittyCacheJCS; import org.nuiton.wikitty.services.WikittyServiceAccessStat.AccessStatStorageWikitty;

1 0

r627 - in trunk: . wikitty-api wikitty-api/src/main/java/org/nuiton/wikitty wikitty-api/src/main/java/org/nuiton/wikitty/services wikitty-api/src/main/resources wikitty-api/src/test/java/org/nuiton/wikitty/layers wikitty-api/src/test/java/org/nuiton/wikitty/services
by bpoussin＠users.nuiton.org 19 Dec '10

19 Dec '10

Author: bpoussin Date: 2010-12-19 04:21:36 +0100 (Sun, 19 Dec 2010) New Revision: 627 Url: http://nuiton.org/repositories/revision/wikitty/627 Log: - refactore cache service to permit use of different cache implementation - add new implementation Cache (use JCS) - fix small syntax error in SERVICE stat access option (SERICE) Added: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheJCS.java trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheSimple.java Modified: trunk/pom.xml trunk/wikitty-api/pom.xml trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCache.java trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceAccessStat.java trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java trunk/wikitty-api/src/main/resources/wikitty-config-sample-client.properties trunk/wikitty-api/src/main/resources/wikitty-config-sample-server-slave.properties trunk/wikitty-api/src/main/resources/wikitty-config-sample-server.properties trunk/wikitty-api/src/main/resources/wikitty-config-sample-standalone.properties trunk/wikitty-api/src/test/java/org/nuiton/wikitty/layers/WikittyServiceCachedTest.java trunk/wikitty-api/src/test/java/org/nuiton/wikitty/services/WikittyServiceCachedTest.java Modified: trunk/pom.xml =================================================================== --- trunk/pom.xml 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/pom.xml 2010-12-19 03:21:36 UTC (rev 627) @@ -37,6 +37,13 @@ <dependencyManagement> <dependencies> +  + <dependency> + <groupId>jcs</groupId> + <artifactId>jcs</artifactId> + <version>1.3</version> + </dependency> +  <dependency> <groupId>net.iharder</groupId> Modified: trunk/wikitty-api/pom.xml =================================================================== --- trunk/wikitty-api/pom.xml 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/pom.xml 2010-12-19 03:21:36 UTC (rev 627) @@ -47,6 +47,12 @@  +  + <dependency> + <groupId>jcs</groupId> + <artifactId>jcs</artifactId> + </dependency> +  <dependency> <groupId>net.iharder</groupId> Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/WikittyConfig.java 2010-12-19 03:21:36 UTC (rev 627) @@ -34,6 +34,8 @@ import org.apache.commons.logging.LogFactory; import org.nuiton.util.ApplicationConfig; import org.nuiton.util.ArgumentsParserException; +import org.nuiton.wikitty.services.WikittyCache; +import org.nuiton.wikitty.services.WikittyCacheJCS; import org.nuiton.wikitty.services.WikittyServiceAccessStat.AccessStatStorageWikitty; /** @@ -223,7 +225,12 @@ + " or org.nuiton.wikitty.services.WikittyServiceInMemoryJdbcSolr)"), WikittyServiceInMemory.class.getName(), String.class, false, false), - WIKITTY_WIKITTYSERICEACCESSSTAT_COMPONENTS( + WIKITTY_WIKITTYSERVICECACHED_COMPONENTS( + "wikitty.WikittyServiceCached.components", + _("Indique le composant a utiliser pour le cache"), + WikittyCacheJCS.class.getName(), String.class, false, false), + + WIKITTY_WIKITTYSERVICEACCESSSTAT_COMPONENTS( "wikitty.WikittyServiceAccessStat.components", _("Indique le composant a utiliser pour le stockage des acces"), AccessStatStorageWikitty.class.getName(), String.class, false, false), @@ -314,8 +321,18 @@ WIKITTY_PROXY_TIME_TO_LOG_WARN( "wikitty.proxy.timeToLog.warn", _("maximum time before send log warn with time consumed"), - "1000", Integer.class, false, false); + "1000", Integer.class, false, false), + JCS_DEFAULT( + "jcs.default", + _("jcs auxiliares to use"), + "", String.class, false, false), + + JCS_DEFAULT_CACHEATTRIBUTES_MAXOBJECTS( + "jcs.default.cacheattributes.MaxObjects", + _("jcs maximum number of items allowed in memory"), + "1000", String.class, false, false); + public String key; public String description; public String defaultValue; Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCache.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCache.java 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCache.java 2010-12-19 03:21:36 UTC (rev 627) @@ -1,208 +1,44 @@ -/* - * #%L - * Wikitty :: api - * - * $Id$ - * $HeadURL$ - * %% - * Copyright (C) 2009 - 2010 CodeLutin, Benjamin Poussin - * %% - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as - * published by the Free Software Foundation, either version 3 of the - * License, or (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Lesser Public License for more details. - * - * You should have received a copy of the GNU General Lesser Public - * License along with this program. If not, see - * <http://www.gnu.org/licenses/lgpl-3.0.html>. - * #L% - */ - package org.nuiton.wikitty.services; import org.nuiton.wikitty.entities.Wikitty; -import org.nuiton.wikitty.services.WikittyEvent; -import org.nuiton.wikitty.services.WikittyListener; -import java.util.Collection; -import java.util.HashMap; -import java.util.Map; -import org.apache.commons.collections.map.ReferenceMap; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.nuiton.util.ApplicationConfig; -import org.nuiton.wikitty.WikittyUtil; - /** - * Cette classe sert a introduire du cache dans wikitty. Elle sert a centraliser - * tous les appels au cache pour pouvoir simplement changer de librairie de - * cache si necessaire, meme si pour l'instant on s'appuie le JDK - * + * * @author poussin * @version $Revision$ * * Last update: $Date$ * by : $Author$ */ -public class WikittyCache implements WikittyListener { +public interface WikittyCache { - static private Log log = LogFactory.getLog(WikittyCache.class); - - /** keys are wikitty ids */ - protected Map<String, Wikitty> wikittyCache; - /** - * - * @param config not used currently but necessary in futur to configure the cache - * Create a soft cache. + * Clear all cache. */ - public WikittyCache(ApplicationConfig config) { - this(true); - } + void clearWikitty(); - /** - * Init cache with specific reference type. - * - * @param soft si vrai utilise un cache avec des reference Soft, sinon - * utilise des references static (utile pour les transactions) - */ - public WikittyCache(boolean soft) { - if (soft) { - wikittyCache = new ReferenceMap(ReferenceMap.HARD, ReferenceMap.SOFT); - } else { - wikittyCache = new HashMap<String, Wikitty>(); - } - } + boolean existsWikitty(String id); - public boolean existsWikitty(String id) { - Object o = wikittyCache.get(id); - boolean result = (o != null); - return result; - } - /** * Return wikitty object if is in the cache, null otherwize. - * + * * @param id * @return wikitty object or null */ - public Wikitty getWikitty(String id) { - Wikitty result = wikittyCache.get(id); - return result; - } + Wikitty getWikitty(String id); /** * Only realy put wikitty in cache, if not in cache or version is newer than * one in cache * @param e */ - public void putWikitty(Wikitty e) { - if (e == null) { - return; - } - Wikitty old = getWikitty(e.getId()); - if (old == null - || WikittyUtil.versionGreaterThan(e.getVersion(), old.getVersion())) { - wikittyCache.put(e.getId(), e); + void putWikitty(Wikitty e); - if (log.isTraceEnabled()) { - log.trace("Replace cached wikitty : new version " + e.getVersion() + - " > old version " + (old == null ? null : old.getVersion())); - } - } - else { - if (log.isTraceEnabled()) { - log.trace("Ignoring putWikittyEvent : new version " + e.getVersion() + " < old version " + old.getVersion()); - } - } - } - /** - * batched put - * @param all - */ - public void putAllWikitty(Collection<Wikitty> all) { - for (Wikitty w : all) { - putWikitty(w); - } - } - - /** * Remove wikitty from cache. - * + * * @param id wikitty id to remove */ - public void removeWikitty(String id) { - wikittyCache.remove(id); - } + void removeWikitty(String id); - /** - * batched remove - * @param ids - */ - public void removeAllWikitty(Collection<String> ids) { - for (String id : ids) { - removeWikitty(id); - } - } - - /** - * Clear all cache. - */ - public void clearWikitty() { - wikittyCache.clear(); - } - - /* - * @see org.nuiton.wikitty.WikittyListener#clearWikitty() - */ - @Override - public void clearWikitty(WikittyEvent event) { - clearWikitty(); - } - - /* - * @see org.nuiton.wikitty.WikittyListener#putWikitty(org.nuiton.wikitty.Wikitty[]) - */ - @Override - public void putWikitty(WikittyEvent event) { - Collection<Wikitty> wikitties = event.getWikitties().values(); - putAllWikitty(wikitties); - } - - /* - * @see org.nuiton.wikitty.WikittyListener#removeWikitty(java.lang.String[]) - */ - @Override - public void removeWikitty(WikittyEvent event) { - for (String id : event.getRemoveDate().keySet()) { - removeWikitty(id); - } - } - - /* - * @see org.nuiton.wikitty.WikittyListener#putExtension(org.nuiton.wikitty.WikittyExtension[]) - */ - @Override - public void putExtension(WikittyEvent event) { - - } - - @Override - public void removeExtension(WikittyEvent event) { - - } - - /* - * @see org.nuiton.wikitty.WikittyListener#clearExtension() - */ - @Override - public void clearExtension(WikittyEvent event) { - - } } Added: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheJCS.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheJCS.java (rev 0) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheJCS.java 2010-12-19 03:21:36 UTC (rev 627) @@ -0,0 +1,135 @@ +/* + * #%L + * Wikitty :: api + * + * $Id: WikittyCacheSimple.java 610 2010-12-14 21:41:35Z tchemit $ + * $HeadURL: http://svn.nuiton.org/svn/wikitty/trunk/wikitty-api/src/main/java/org/nuito… $ + * %% + * Copyright (C) 2009 - 2010 CodeLutin, Benjamin Poussin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Lesser Public License for more details. + * + * You should have received a copy of the GNU General Lesser Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/lgpl-3.0.html>. + * #L% + */ + +package org.nuiton.wikitty.services; + +import org.nuiton.wikitty.entities.Wikitty; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.jcs.JCS; +import org.apache.jcs.access.GroupCacheAccess; +import org.apache.jcs.access.exception.CacheException; +import org.apache.jcs.engine.control.CompositeCacheManager; +import org.nuiton.util.ApplicationConfig; +import org.nuiton.wikitty.WikittyException; + +/** + * Implantation du cache base sur le projet apache JCS + * + * @author poussin + * @version $Revision: 610 $ + * + * Last update: $Date: 2010-12-14 22:41:35 +0100 (mar. 14 déc. 2010) $ + * by : $Author: tchemit $ + */ +public class WikittyCacheJCS implements WikittyCache { + + static private Log log = LogFactory.getLog(WikittyCacheJCS.class); + + protected GroupCacheAccess wikittyCache; + + /** + * + * @param config not used currently but necessary in futur to configure the cache + * Create a soft cache. + */ + public WikittyCacheJCS(ApplicationConfig config) { + try { + CompositeCacheManager cacheMgr = CompositeCacheManager.getUnconfiguredInstance(); + cacheMgr.configure(config.getFlatOptions()); + wikittyCache = new GroupCacheAccess(cacheMgr.getCache("wikitty")); + + // JCS ne permet pas de passer un Properties mais seulement un nom de fichier :( +// JCS.setConfigFilename(config.getConfigFileName()); +// wikittyCache = JCS.getInstance("wikitty"); + } catch (Exception eee) { + throw new WikittyException("Can't initialise JCS cache", eee); + } + } + + + @Override + public boolean existsWikitty(String id) { + Object o = wikittyCache.get(id); + boolean result = (o != null); + return result; + } + + /** + * Return wikitty object if is in the cache, null otherwize. + * + * @param id + * @return wikitty object or null + */ + @Override + public Wikitty getWikitty(String id) { + Wikitty result = (Wikitty)wikittyCache.get(id); + return result; + } + + /** + * Only realy put wikitty in cache, if not in cache or version is newer than + * one in cache + * @param e + */ + @Override + public void putWikitty(Wikitty e) { + if (e != null) { + try { + wikittyCache.put(e.getId(), e); + } catch (CacheException eee) { + log.error(String.format("Can't put wikitty %s in cache", e), eee); + } + } + } + + /** + * Remove wikitty from cache. + * + * @param id wikitty id to remove + */ + @Override + public void removeWikitty(String id) { + try { + wikittyCache.remove(id); + } catch (CacheException eee) { + log.error(String.format("Can't remove wikitty %s in cache", id), eee); + } + } + + /** + * Clear all cache. + */ + @Override + public void clearWikitty() { + try { + wikittyCache.clear(); + } catch (CacheException eee) { + log.error(String.format("Can't clear wikitty cache"), eee); + } + } + +} Copied: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheSimple.java (from rev 610, trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCache.java) =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheSimple.java (rev 0) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyCacheSimple.java 2010-12-19 03:21:36 UTC (rev 627) @@ -0,0 +1,127 @@ +/* + * #%L + * Wikitty :: api + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2009 - 2010 CodeLutin, Benjamin Poussin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Lesser Public License for more details. + * + * You should have received a copy of the GNU General Lesser Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/lgpl-3.0.html>. + * #L% + */ + +package org.nuiton.wikitty.services; + +import org.nuiton.wikitty.entities.Wikitty; +import java.util.HashMap; +import java.util.Map; + +import org.apache.commons.collections.map.ReferenceMap; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.nuiton.util.ApplicationConfig; + +/** + * Cette classe sert a introduire du cache dans wikitty. Elle sert a centraliser + * tous les appels au cache pour pouvoir simplement changer de librairie de + * cache si necessaire, meme si pour l'instant on s'appuie le JDK + * + * @author poussin + * @version $Revision$ + * + * Last update: $Date$ + * by : $Author$ + */ +public class WikittyCacheSimple implements WikittyCache { + + static private Log log = LogFactory.getLog(WikittyCacheSimple.class); + + /** keys are wikitty ids */ + protected Map<String, Wikitty> wikittyCache; + + /** + * + * @param config not used currently but necessary in futur to configure the cache + * Create a soft cache. + */ + public WikittyCacheSimple(ApplicationConfig config) { + this(true); + } + + /** + * Init cache with specific reference type. + * + * @param soft si vrai utilise un cache avec des reference Soft, sinon + * utilise des references static (utile pour les transactions) + */ + public WikittyCacheSimple(boolean soft) { + if (soft) { + wikittyCache = new ReferenceMap(ReferenceMap.HARD, ReferenceMap.SOFT); + } else { + wikittyCache = new HashMap<String, Wikitty>(); + } + } + + @Override + public boolean existsWikitty(String id) { + Object o = wikittyCache.get(id); + boolean result = (o != null); + return result; + } + + /** + * Return wikitty object if is in the cache, null otherwize. + * + * @param id + * @return wikitty object or null + */ + @Override + public Wikitty getWikitty(String id) { + Wikitty result = wikittyCache.get(id); + return result; + } + + /** + * Only realy put wikitty in cache, if not in cache or version is newer than + * one in cache + * @param e + */ + @Override + public void putWikitty(Wikitty e) { + if (e != null) { + wikittyCache.put(e.getId(), e); + } + } + + /** + * Remove wikitty from cache. + * + * @param id wikitty id to remove + */ + @Override + public void removeWikitty(String id) { + wikittyCache.remove(id); + } + + /** + * Clear all cache. + */ + @Override + public void clearWikitty() { + wikittyCache.clear(); + } + +} Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceAccessStat.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceAccessStat.java 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceAccessStat.java 2010-12-19 03:21:36 UTC (rev 627) @@ -84,7 +84,7 @@ if (statStorage == null) { if (log.isWarnEnabled()) { String statSto = config.getOption(WikittyConfig.WikittyOption. - WIKITTY_WIKITTYSERICEACCESSSTAT_COMPONENTS.getKey()); + WIKITTY_WIKITTYSERVICEACCESSSTAT_COMPONENTS.getKey()); log.warn(String.format( "No AccessStatStorage available, access stat can't work (%s)", statSto)); Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java 2010-12-19 03:21:36 UTC (rev 627) @@ -45,6 +45,7 @@ import org.nuiton.wikitty.entities.WikittyTreeNode; import org.nuiton.wikitty.WikittyService; import org.nuiton.wikitty.WikittyTree; +import org.nuiton.wikitty.WikittyUtil; /** * Override some method of WikittyService to use cache @@ -63,6 +64,9 @@ /** Cache. */ protected WikittyCache cache = null; + /** pour recevoir les events du serveur et synchroniser le cache */ + protected RemoteWikittyListener remoteWikittyListener; + /** Delegated wikitty service implementation. */ protected WikittyService ws; @@ -89,67 +93,153 @@ * @param ws delegate service * @param props properties (can be null) */ - public WikittyServiceCached(ApplicationConfig config, WikittyService ws) { + public WikittyServiceCached(ApplicationConfig config, WikittyService ws, WikittyCache cache) { this.ws = ws; - cache = new WikittyCache(config); + this.cache = cache; + if (cache == null) { + if (log.isWarnEnabled()) { + String cacheClassName = config.getOption(WikittyConfig.WikittyOption. + WIKITTY_WIKITTYSERVICECACHED_COMPONENTS.getKey()); + log.warn(String.format( + "No cache class implementation available (%s), use simple cache", + cacheClassName)); + } + this.cache = new WikittyCacheSimple(config); + } + + // add service listener for synchronisation listener for remote event // register all time, but perhaps Notifier service don't listen for // remote event and cache can't be notified. But this is normal // behavior try { - addWikittyServiceListener(cache, ServiceListenerType.REMOTE); + remoteWikittyListener = new RemoteWikittyListener(this); + addWikittyServiceListener( + remoteWikittyListener, ServiceListenerType.REMOTE); } catch(UnsupportedOperationException eee) { - log.info("no WikittyServiceNotifier available, cache don't listen event"); + log.warn("no WikittyServiceNotifier available, cache don't listen event"); } if (config != null) { // reading configuration and set allwaysRestoreCopies accordingly allwaysRestoreCopies = - config.getOptionAsBoolean(WikittyConfig.WikittyOption.WIKITTY_CACHE_RESTORE_COPIES.getKey()); + config.getOptionAsBoolean( + WikittyConfig.WikittyOption.WIKITTY_CACHE_RESTORE_COPIES.getKey()); } } protected void statAdd(int asked, int missed) { this.asked += asked; this.missed += missed; - if(log.isDebugEnabled() || true) { +// if(log.isDebugEnabled()) { String total = StringUtil.convertMemory(Runtime.getRuntime().totalMemory()); String free = StringUtil.convertMemory(Runtime.getRuntime().freeMemory()); String msg = String.format("cache stat (missed/asked): %s/%s (memory %s/%s [total/free])", this.missed, this.asked, total, free); log.debug(msg); - } + System.out.println(msg); +// } } /** wrap the wikitty or copy it according to allwaysRestoreCopies value */ protected Wikitty wrapWikitty(Wikitty wikitty) { + Wikitty result = null; // Restored wikitty can be null - if (wikitty == null) { - return null; + if (wikitty != null) { + if (allwaysRestoreCopies) { + try { + result = wikitty.clone(); + } catch (CloneNotSupportedException eee) { + // si on arrive pas a faire un clone, on retourne null + // comme si l'objet n'etait pas dans le cache, pour que + // l'application fonctionne tout de meme mais en mode + // degrade (sans cache) + log.error(String.format( + "Cache doesn't work, unable to clone %s", wikitty), eee); + } + } else { + // normalement, on a que des WikittyImpl ici, mais on fait + // un petit check pour etre sur qu'on ne wrap pas un object wrappe + if (wikitty instanceof WikittyCopyOnWrite) { + wikitty = ((WikittyCopyOnWrite)wikitty).getTarget(); + } + result = new WikittyCopyOnWrite(wikitty); + } } + return result; + } - Wikitty result = null; - if (allwaysRestoreCopies) { - try { - result = wikitty.clone(); - } catch (CloneNotSupportedException e) { - log.error("unable to clone " + wikitty, e); + // + // seulement les methodes suivantes doivent acceder au cache pour centraliser + // son access + // + + protected void clearCache() { + cache.clearWikitty(); + } + + /** + * Only WikittyImpl can be put in real cache implementation. If argument + * is WikittyCopyOnWrite, we must take internal wikitty to put in cache + */ + protected void putInCache(Wikitty w) { + if (w != null) { + if (w instanceof WikittyCopyOnWrite) { + w = ((WikittyCopyOnWrite) w).getTarget(); } + + Wikitty old = getFromCache(w.getId()); + if (old == null + || WikittyUtil.versionGreaterThan(w.getVersion(), old.getVersion())) { + cache.putWikitty(w); + + if (log.isTraceEnabled()) { + log.trace("Replace cached wikitty : new version " + w.getVersion() + + " > old version " + (old == null ? null : old.getVersion())); + } + } else { + if (log.isTraceEnabled()) { + log.trace(String.format( + "Ignoring putWikittyEvent : new version %s < old version %s", + w.getVersion(), old.getVersion())); + } + } } - - // if allwaysRestoreCopies is false and above clone failed - if (result == null) { - result = new WikittyCopyOnWrite(wikitty); + } + + protected void putInCache(Collection<Wikitty> wikitties) { + for (Wikitty w : wikitties) { + putInCache(w); } + } + protected void removeInCache(String id) { + cache.removeWikitty(id); + } + + protected void removeInCache(Collection<String> ids) { + for (String id : ids) { + removeInCache(id); + } + } + + protected Wikitty getFromCache(String id) { + Wikitty result = cache.getWikitty(id); + + // all time wrap, inly WikittyImpl are in cache + result = wrapWikitty(result); return result; } + + // + // surcharge des methodes du WikittyService + // @Override public WikittyEvent clear(String securityToken) { WikittyEvent result = ws.clear(securityToken); - cache.clearWikitty(); + clearCache(); return result; } @@ -160,7 +250,7 @@ @Override public WikittyEvent delete(String securityToken, Collection<String> ids) { WikittyEvent result = ws.delete(securityToken, ids); - cache.removeAllWikitty(ids); + removeInCache(ids); return result; } @@ -232,7 +322,7 @@ LinkedHashMap<String, Wikitty> fromCache = new LinkedHashMap<String, Wikitty>(); for (String id : ids) { - Wikitty w = cache.getWikitty(id); + Wikitty w = getFromCache(id); fromCache.put(id, w); // put all to maintains order if (w == null) { // if not found on cache, ask the server notInCache.add(id); @@ -242,7 +332,7 @@ // retrieve missing object List<Wikitty> missingInCache = ws.restore(securityToken, notInCache); - cache.putAllWikitty(missingInCache); + putInCache(missingInCache); for (Wikitty w : missingInCache) { // add missing object @@ -343,7 +433,7 @@ Map<String, Wikitty> wikittiesToCache = result.getWikitties(); if (wikittiesToCache != null) { - cache.putAllWikitty(wikittiesToCache.values()); + putInCache(wikittiesToCache.values()); } return result; @@ -436,4 +526,64 @@ return ws.isDeleted(securityToken, wikittyId); } + + /** + * Classe permettant de recevoir les events distants et mettre a jour le cache + */ + static public class RemoteWikittyListener implements WikittyListener { + + protected WikittyServiceCached wsCached; + + public RemoteWikittyListener(WikittyServiceCached wsCached) { + this.wsCached = wsCached; + } + + /* + * @see org.nuiton.wikitty.WikittyListener#clearWikitty() + */ + @Override + public void clearWikitty(WikittyEvent event) { + wsCached.clearCache(); + } + + /* + * @see org.nuiton.wikitty.WikittyListener#putWikitty(org.nuiton.wikitty.Wikitty[]) + */ + @Override + public void putWikitty(WikittyEvent event) { + Collection<Wikitty> wikitties = event.getWikitties().values(); + wsCached.putInCache(wikitties); + } + + /* + * @see org.nuiton.wikitty.WikittyListener#removeWikitty(java.lang.String[]) + */ + @Override + public void removeWikitty(WikittyEvent event) { + if (event.getRemoveDate() != null) { + Collection<String> ids = event.getRemoveDate().keySet(); + wsCached.removeInCache(ids); + } + } + + /* + * @see org.nuiton.wikitty.WikittyListener#putExtension(org.nuiton.wikitty.WikittyExtension[]) + */ + @Override + public void putExtension(WikittyEvent event) { + } + + @Override + public void removeExtension(WikittyEvent event) { + } + + /* + * @see org.nuiton.wikitty.WikittyListener#clearExtension() + */ + @Override + public void clearExtension(WikittyEvent event) { + wsCached.clearCache(); + } + + } } Modified: trunk/wikitty-api/src/main/resources/wikitty-config-sample-client.properties =================================================================== --- trunk/wikitty-api/src/main/resources/wikitty-config-sample-client.properties 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/src/main/resources/wikitty-config-sample-client.properties 2010-12-19 03:21:36 UTC (rev 627) @@ -27,9 +27,12 @@ org.nuiton.wikitty.services.WikittyServiceCached,\ org.nuiton.wikitty.services.WikittyServiceSecurity wikitty.WikittyServiceNotifier.components=org.nuiton.wikitty.services.XMPPNotifierTransporter +wikitty.WikittyServiceCached.components=org.nuiton.wikitty.services.WikittyCacheJCS wikitty.service.cache.allwaysRestoreCopies=false wikitty.service.event.propagate=false wikitty.service.event.listen=true wikitty.service.event.transporter.xmpp.server=im.codelutin.com wikitty.service.event.transporter.xmpp.room=master(a)conference.im.codelutin.com wikitty.service.server.url=http://services.codelutin.com/wikitty +jcs.default= +jcs.default.cacheattributes.MaxObjects=1000 Modified: trunk/wikitty-api/src/main/resources/wikitty-config-sample-server-slave.properties =================================================================== --- trunk/wikitty-api/src/main/resources/wikitty-config-sample-server-slave.properties 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/src/main/resources/wikitty-config-sample-server-slave.properties 2010-12-19 03:21:36 UTC (rev 627) @@ -44,6 +44,7 @@ org.nuiton.wikitty.jdbc.WikittyStorageJDBC,\ org.nuiton.wikitty.storage.solr.WikittySearchEngineSolr wikitty.WikittyServiceNotifier.components=org.nuiton.wikitty.services.XMPPNotifierTransporter +wikitty.WikittyServiceCached.components=org.nuiton.wikitty.services.WikittyCacheJCS wikitty.service.server.url=http://services-slave.codelutin.com/wikitty wikitty.service.cache.allwaysRestoreCopies=false wikitty.service.event.propagate=true @@ -57,3 +58,5 @@ wikitty.addon.export.threadnumber=1 wikitty.addon.export.directory=${wikitty.data.directory}/export wikitty.addon.export.publicurl=file://${wikitty.data.directory}/export/ +jcs.default= +jcs.default.cacheattributes.MaxObjects=1000 Modified: trunk/wikitty-api/src/main/resources/wikitty-config-sample-server.properties =================================================================== --- trunk/wikitty-api/src/main/resources/wikitty-config-sample-server.properties 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/src/main/resources/wikitty-config-sample-server.properties 2010-12-19 03:21:36 UTC (rev 627) @@ -43,6 +43,7 @@ org.nuiton.wikitty.jdbc.WikittyStorageJDBC,\ org.nuiton.wikitty.storage.solr.WikittySearchEngineSolr wikitty.WikittyServiceNotifier.components=org.nuiton.wikitty.services.XMPPNotifierTransporter +wikitty.WikittyServiceCached.components=org.nuiton.wikitty.services.WikittyCacheJCS wikitty.service.server.url=http://services.codelutin.com/wikitty wikitty.service.cache.listenevents=false wikitty.service.cache.allwaysRestoreCopies=false @@ -53,3 +54,5 @@ wikitty.addon.export.threadnumber=1 wikitty.addon.export.directory=${wikitty.data.directory}/export wikitty.addon.export.publicurl=file://${wikitty.data.directory}/export +jcs.default= +jcs.default.cacheattributes.MaxObjects=1000 Modified: trunk/wikitty-api/src/main/resources/wikitty-config-sample-standalone.properties =================================================================== --- trunk/wikitty-api/src/main/resources/wikitty-config-sample-standalone.properties 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/src/main/resources/wikitty-config-sample-standalone.properties 2010-12-19 03:21:36 UTC (rev 627) @@ -41,6 +41,7 @@ wikitty.WikittyServiceStorage.components=org.nuiton.wikitty.jdbc.WikittyExtensionStorageJDBC,\ org.nuiton.wikitty.jdbc.WikittyStorageJDBC,\ org.nuiton.wikitty.storage.solr.WikittySearchEngineSolr +wikitty.WikittyServiceCached.components=org.nuiton.wikitty.services.WikittyCacheJCS wikitty.service.cache.listenevents=false wikitty.service.cache.allwaysRestoreCopies=false wikitty.service.event.propagate=false @@ -48,3 +49,5 @@ wikitty.addon.export.threadnumber=1 wikitty.addon.export.directory=${wikitty.data.directory}/export wikitty.addon.export.publicurl=file://${wikitty.data.directory}/export/ +jcs.default= +jcs.default.cacheattributes.MaxObjects=1000 Modified: trunk/wikitty-api/src/test/java/org/nuiton/wikitty/layers/WikittyServiceCachedTest.java =================================================================== --- trunk/wikitty-api/src/test/java/org/nuiton/wikitty/layers/WikittyServiceCachedTest.java 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/src/test/java/org/nuiton/wikitty/layers/WikittyServiceCachedTest.java 2010-12-19 03:21:36 UTC (rev 627) @@ -35,6 +35,7 @@ import org.junit.Test; import org.nuiton.wikitty.entities.Wikitty; import org.nuiton.wikitty.WikittyConfig; +import org.nuiton.wikitty.services.WikittyCacheJCS; import org.nuiton.wikitty.services.WikittyServiceCached; import org.nuiton.wikitty.services.WikittyServiceInMemory; @@ -44,7 +45,9 @@ @Before public void setUpWikittyServiceCachedTest() { WikittyConfig config = new WikittyConfig(); - setService(new WikittyServiceCached(config, new WikittyServiceInMemory(config))); + setService(new WikittyServiceCached(config, + new WikittyServiceInMemory(config), + new WikittyCacheJCS(config))); token = service.login(null, null); service.store(token, Collections.singletonList(getaWikitty()), false); } @@ -96,7 +99,9 @@ WikittyConfig.WikittyOption.WIKITTY_CACHE_RESTORE_COPIES.getKey(), "true"); - setService(new WikittyServiceCached(config, new WikittyServiceInMemory(config))); + setService(new WikittyServiceCached(config, + new WikittyServiceInMemory(config), + new WikittyCacheJCS(config))); token = service.login(null, null); service.store(token, Collections.singletonList(getaWikitty()), false); Modified: trunk/wikitty-api/src/test/java/org/nuiton/wikitty/services/WikittyServiceCachedTest.java =================================================================== --- trunk/wikitty-api/src/test/java/org/nuiton/wikitty/services/WikittyServiceCachedTest.java 2010-12-17 17:51:06 UTC (rev 626) +++ trunk/wikitty-api/src/test/java/org/nuiton/wikitty/services/WikittyServiceCachedTest.java 2010-12-19 03:21:36 UTC (rev 627) @@ -34,10 +34,13 @@ @Test public void testCache() throws Exception { ApplicationConfig config = getConfig(); + config.setOption(WikittyConfig.WikittyOption + .JCS_DEFAULT_CACHEATTRIBUTES_MAXOBJECTS.getKey(), "10"); WikittyServiceInMemory ws = new WikittyServiceInMemory(config); - WikittyServiceCached cache = new WikittyServiceCached(config, ws); + WikittyCache cache = new WikittyCacheJCS(config); + WikittyServiceCached wscached = new WikittyServiceCached(config, ws, cache); - WikittyProxy proxy = new WikittyProxy(config, cache); + WikittyProxy proxy = new WikittyProxy(config, wscached); List<BusinessEntity> toStore = new ArrayList<BusinessEntity>(); List<String> toRestore = new ArrayList<String>();

1 0

r626 - in trunk/wikitty-api/src: main/java/org/nuiton/wikitty/services test/java/org/nuiton/wikitty/services
by bpoussin＠users.nuiton.org 17 Dec '10

17 Dec '10

Author: bpoussin Date: 2010-12-17 18:51:06 +0100 (Fri, 17 Dec 2010) New Revision: 626 Url: http://nuiton.org/repositories/revision/wikitty/626 Log: - add memory usage in cache log - add small test of cache service Added: trunk/wikitty-api/src/test/java/org/nuiton/wikitty/services/WikittyServiceCachedTest.java Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java Modified: trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java =================================================================== --- trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java 2010-12-17 17:18:17 UTC (rev 625) +++ trunk/wikitty-api/src/main/java/org/nuiton/wikitty/services/WikittyServiceCached.java 2010-12-17 17:51:06 UTC (rev 626) @@ -35,6 +35,7 @@ import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.nuiton.util.ApplicationConfig; +import org.nuiton.util.StringUtil; import org.nuiton.wikitty.search.Criteria; import org.nuiton.wikitty.search.PagedResult; import org.nuiton.wikitty.entities.Wikitty; @@ -112,8 +113,12 @@ protected void statAdd(int asked, int missed) { this.asked += asked; this.missed += missed; - if(log.isDebugEnabled()) { - log.debug(String.format("cache stat (missed/asked): %s/%s", missed, asked)); + if(log.isDebugEnabled() || true) { + String total = StringUtil.convertMemory(Runtime.getRuntime().totalMemory()); + String free = StringUtil.convertMemory(Runtime.getRuntime().freeMemory()); + String msg = String.format("cache stat (missed/asked): %s/%s (memory %s/%s [total/free])", + this.missed, this.asked, total, free); + log.debug(msg); } } Added: trunk/wikitty-api/src/test/java/org/nuiton/wikitty/services/WikittyServiceCachedTest.java =================================================================== --- trunk/wikitty-api/src/test/java/org/nuiton/wikitty/services/WikittyServiceCachedTest.java (rev 0) +++ trunk/wikitty-api/src/test/java/org/nuiton/wikitty/services/WikittyServiceCachedTest.java 2010-12-17 17:51:06 UTC (rev 626) @@ -0,0 +1,75 @@ +package org.nuiton.wikitty.services; + + +import java.util.ArrayList; +import java.util.List; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Test; +import org.nuiton.util.ApplicationConfig; +import org.nuiton.wikitty.WikittyConfig; +import org.nuiton.wikitty.WikittyProxy; +import org.nuiton.wikitty.entities.BusinessEntity; +import org.nuiton.wikitty.entities.WikittyUser; +import org.nuiton.wikitty.entities.WikittyUserImpl; + +/** + * + * @author poussin + * @version $Revision$ + * + * Last update: $Date$ + * by : $Author$ + */ +public class WikittyServiceCachedTest { + + /** to use log facility, just put in your code: log.info(\"...\"); */ + static private Log log = LogFactory.getLog(WikittyServiceCachedTest.class); + + protected ApplicationConfig getConfig() { + WikittyConfig result = new WikittyConfig(); + return result; + } + + @Test + public void testCache() throws Exception { + ApplicationConfig config = getConfig(); + WikittyServiceInMemory ws = new WikittyServiceInMemory(config); + WikittyServiceCached cache = new WikittyServiceCached(config, ws); + + WikittyProxy proxy = new WikittyProxy(config, cache); + List<BusinessEntity> toStore = new ArrayList<BusinessEntity>(); + List<String> toRestore = new ArrayList<String>(); + + WikittyUser u1 = new WikittyUserImpl(); + WikittyUser u2 = new WikittyUserImpl(); + WikittyUser u3 = new WikittyUserImpl(); + WikittyUser u4 = new WikittyUserImpl(); + WikittyUser u5 = new WikittyUserImpl(); + + toStore.add(u1); + toStore.add(u2); + toStore.add(u3); + toStore.add(u4); + toStore.add(u5); + + toStore = proxy.store(toStore); + + u1 = proxy.restore(WikittyUser.class, u1.getWikittyId()); + u2 = proxy.restore(WikittyUser.class, u2.getWikittyId()); + u3 = proxy.restore(WikittyUser.class, u3.getWikittyId()); + u4 = proxy.restore(WikittyUser.class, u4.getWikittyId()); + u5 = proxy.restore(WikittyUser.class, u5.getWikittyId()); + u1 = proxy.restore(WikittyUser.class, u1.getWikittyId()); + u2 = proxy.restore(WikittyUser.class, u2.getWikittyId()); + u3 = proxy.restore(WikittyUser.class, u3.getWikittyId()); + + toRestore.add(u1.getWikittyId()); + toRestore.add(u2.getWikittyId()); + toRestore.add(u3.getWikittyId()); + toRestore.add(u4.getWikittyId()); + toRestore.add(u5.getWikittyId()); + + proxy.restore(WikittyUser.class, toRestore); + } +}

1 0

r625 - trunk/wikitty-solr-impl/src/main/resources
by bpoussin＠users.nuiton.org 17 Dec '10

17 Dec '10

Author: bpoussin Date: 2010-12-17 18:18:17 +0100 (Fri, 17 Dec 2010) New Revision: 625 Url: http://nuiton.org/repositories/revision/wikitty/625 Log: remove unused file Removed: trunk/wikitty-solr-impl/src/main/resources/solrconfig-complete.xml trunk/wikitty-solr-impl/src/main/resources/stopwords.txt Deleted: trunk/wikitty-solr-impl/src/main/resources/solrconfig-complete.xml =================================================================== --- trunk/wikitty-solr-impl/src/main/resources/solrconfig-complete.xml 2010-12-17 17:10:51 UTC (rev 624) +++ trunk/wikitty-solr-impl/src/main/resources/solrconfig-complete.xml 2010-12-17 17:18:17 UTC (rev 625) @@ -1,733 +0,0 @@ -<?xml version="1.0" encoding="UTF-8" ?> - - - - -<config> -  - <abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError> - -  - <dataDir>${wikitty.searchengine.solr.directory.data:./solr/data}</dataDir> - -  - <directoryFactory name="DirectoryFactory" class="${wikitty.searchengine.solr.directory.factory:solr.StandardDirectoryFactory}"/> - - <indexDefaults> -  - <useCompoundFile>false</useCompoundFile> - - <mergeFactor>10</mergeFactor> -  -  -  - <ramBufferSizeMB>32</ramBufferSizeMB> - <maxMergeDocs>2147483647</maxMergeDocs> - <maxFieldLength>10000</maxFieldLength> - <writeLockTimeout>1000</writeLockTimeout> - <commitLockTimeout>10000</commitLockTimeout> - -  -  -  -  - -  -  - -  - <lockType>simple</lockType> - </indexDefaults> - - <mainIndex> -  - <useCompoundFile>false</useCompoundFile> - <ramBufferSizeMB>32</ramBufferSizeMB> - <mergeFactor>10</mergeFactor> -  -  - <maxMergeDocs>2147483647</maxMergeDocs> - <maxFieldLength>10000</maxFieldLength> - -  - <unlockOnStartup>true</unlockOnStartup> - </mainIndex> - -  - <jmx /> - -  - <updateHandler class="solr.DirectUpdateHandler2"> - -  - -  - -  -  -  - - </updateHandler> - - - <query> -  - <maxBooleanClauses>1024</maxBooleanClauses> - - -  - <filterCache - class="solr.LRUCache" - size="512" - initialSize="512" - autowarmCount="128"/> - -  - <queryResultCache - class="solr.LRUCache" - size="512" - initialSize="512" - autowarmCount="32"/> - -  - <documentCache - class="solr.LRUCache" - size="512" - initialSize="512" - autowarmCount="0"/> - -  - <enableLazyFieldLoading>true</enableLazyFieldLoading> - -  -  - -  - -  - <queryResultWindowSize>50</queryResultWindowSize> - -  - <queryResultMaxDocsCached>200</queryResultMaxDocsCached> - -  - <HashDocSet maxSize="3000" loadFactor="0.75"/> - -  -  - <listener event="newSearcher" class="solr.QuerySenderListener"> - <arr name="queries"> - <lst> <str name="q">solr</str> <str name="start">0</str> <str name="rows">10</str> </lst> - <lst> <str name="q">rocks</str> <str name="start">0</str> <str name="rows">10</str> </lst> - <lst><str name="q">static newSearcher warming query from solrconfig.xml</str></lst> - </arr> - </listener> - -  - <listener event="firstSearcher" class="solr.QuerySenderListener"> - <arr name="queries"> - <lst> <str name="q">fast_warm</str> <str name="start">0</str> <str name="rows">10</str> </lst> - <lst><str name="q">static firstSearcher warming query from solrconfig.xml</str></lst> - </arr> - </listener> - -  - <useColdSearcher>false</useColdSearcher> - -  - <maxWarmingSearchers>2</maxWarmingSearchers> - - </query> - -  - <requestDispatcher handleSelect="true" > -  - <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" /> - -  -  - <httpCaching lastModifiedFrom="openTime" - etagSeed="Solr"> -  -  -  - </httpCaching> - </requestDispatcher> - - -  - <requestHandler name="standard" class="solr.SearchHandler" default="true"> -  - <lst name="defaults"> - <str name="echoParams">explicit</str> -  - </lst> - </requestHandler> - - -  - <requestHandler name="dismax" class="solr.SearchHandler" > - <lst name="defaults"> - <str name="defType">dismax</str> - <str name="echoParams">explicit</str> - <float name="tie">0.01</float> - <str name="qf"> - text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4 - </str> - <str name="pf"> - text^0.2 features^1.1 name^1.5 manu^1.4 manu_exact^1.9 - </str> - <str name="bf"> - ord(popularity)^0.5 recip(rord(price),1,1000,1000)^0.3 - </str> - <str name="fl"> - id,name,price,score - </str> - <str name="mm"> - 2<-1 5<-2 6<90% - </str> - <int name="ps">100</int> - <str name="q.alt">*:*</str> -  - <str name="hl.fl">text features name</str> -  - <str name="f.name.hl.fragsize">0</str> -  - <str name="f.name.hl.alternateField">name</str> - <str name="f.text.hl.fragmenter">regex</str>  - </lst> - </requestHandler> - -  - <requestHandler name="partitioned" class="solr.SearchHandler" > - <lst name="defaults"> - <str name="defType">dismax</str> - <str name="echoParams">explicit</str> - <str name="qf">text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0</str> - <str name="mm">2<-1 5<-2 6<90%</str> -  - <str name="bq">incubationdate_dt:[* TO NOW/DAY-1MONTH]^2.2</str> - </lst> -  - <lst name="appends"> - <str name="fq">inStock:true</str> - </lst> -  - <lst name="invariants"> - <str name="facet.field">cat</str> - <str name="facet.field">manu_exact</str> - <str name="facet.query">price:[* TO 500]</str> - <str name="facet.query">price:[500 TO *]</str> - </lst> - </requestHandler> - - -  - -  - <searchComponent name="spellcheck" class="solr.SpellCheckComponent"> - - <str name="queryAnalyzerFieldType">textSpell</str> - - <lst name="spellchecker"> - <str name="name">default</str> - <str name="field">spell</str> - <str name="spellcheckIndexDir">./spellchecker1</str> - - </lst> - <lst name="spellchecker"> - <str name="name">jarowinkler</str> - <str name="field">spell</str> -  - <str name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance</str> - <str name="spellcheckIndexDir">./spellchecker2</str> - - </lst> - - <lst name="spellchecker"> - <str name="classname">solr.FileBasedSpellChecker</str> - <str name="name">file</str> - <str name="sourceLocation">spellings.txt</str> - <str name="characterEncoding">UTF-8</str> - <str name="spellcheckIndexDir">./spellcheckerFile</str> - </lst> - </searchComponent> - -  - <requestHandler name="/spellCheckCompRH" class="solr.SearchHandler"> - <lst name="defaults"> -  - <str name="spellcheck.onlyMorePopular">false</str> -  - <str name="spellcheck.extendedResults">false</str> -  - <str name="spellcheck.count">1</str> - </lst> - <arr name="last-components"> - <str>spellcheck</str> - </arr> - </requestHandler> - -  - - -  - - -  - <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" /> - -  - <requestHandler name="/analysis" class="solr.AnalysisRequestHandler" /> - - -  - <requestHandler name="/update/csv" class="solr.CSVRequestHandler" startup="lazy" /> - - -  - <requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" /> - -  - <requestHandler name="/admin/ping" class="PingRequestHandler"> - <lst name="defaults"> - <str name="qt">standard</str> - <str name="q">solrpingquery</str> - <str name="echoParams">all</str> - </lst> - </requestHandler> - -  - <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" > - <lst name="defaults"> - <str name="echoParams">explicit</str>  - <str name="echoHandler">true</str> - </lst> - </requestHandler> - - <highlighting> -  -  - <fragmenter name="gap" class="org.apache.solr.highlight.GapFragmenter" default="true"> - <lst name="defaults"> - <int name="hl.fragsize">100</int> - </lst> - </fragmenter> - -  - <fragmenter name="regex" class="org.apache.solr.highlight.RegexFragmenter"> - <lst name="defaults"> -  - <int name="hl.fragsize">70</int> -  - <float name="hl.regex.slop">0.5</float> -  - <str name="hl.regex.pattern">[-\w ,/\n\"']{20,200}</str> - </lst> - </fragmenter> - -  - <formatter name="html" class="org.apache.solr.highlight.HtmlFormatter" default="true"> - <lst name="defaults"> - <str name="hl.simple.pre"><![CDATA[<em>]]></str> - <str name="hl.simple.post"><![CDATA[</em>]]></str> - </lst> - </formatter> - </highlighting> - - -  - -  - <queryResponseWriter name="xslt" class="org.apache.solr.request.XSLTResponseWriter"> - <int name="xsltCacheLifetimeSeconds">5</int> - </queryResponseWriter> - - - <queryParser name="wikitty" class="org.nuiton.wikitty.solr.WikittyQueryParser"/> - -  - -  - -  - <admin> - <defaultQuery>solr</defaultQuery> - -  - </admin> - -</config> Deleted: trunk/wikitty-solr-impl/src/main/resources/stopwords.txt =================================================================== --- trunk/wikitty-solr-impl/src/main/resources/stopwords.txt 2010-12-17 17:10:51 UTC (rev 624) +++ trunk/wikitty-solr-impl/src/main/resources/stopwords.txt 2010-12-17 17:18:17 UTC (rev 625) @@ -1,57 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -#----------------------------------------------------------------------- -# a couple of test stopwords to test that the words are really being -# configured from this file: -stopworda -stopwordb - -#Standard english stop words taken from Lucene's StopAnalyzer -an -and -are -as -at -be -but -by -for -if -in -into -is -it -no -not -of -on -or -s -such -t -that -the -their -then -there -these -they -this -to -was -will -with -

1 0