- 
                Notifications
    
You must be signed in to change notification settings  - Fork 2.2k
 
#4353 Plugin for auto-generated sitemaps #4413
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Merged
      
      
            aldeed
  merged 43 commits into
  release-1.15.0
from
feat-4353-dancastellon-sitemap-generator
  
      
      
   
  Aug 16, 2018 
      
    
  
     Merged
                    Changes from all commits
      Commits
    
    
            Show all changes
          
          
            43 commits
          
        
        Select commit
          Hold shift + click to select a range
      
      c7a0d51
              
                (feat): generate sitemaps on demand and store in collection
              
              
                dancastellon f671b6c
              
                (style): correct linting issues
              
              
                dancastellon 59226ad
              
                (feat): Routes for auto-generated sitemap files
              
              
                dancastellon db2d035
              
                (feat): Add compound index for Sitemaps queries
              
              
                dancastellon 56de04e
              
                (refactor): Cleanup generateSitemaps function
              
              
                dancastellon b423901
              
                (style): corrent linting issues
              
              
                dancastellon a7a11c2
              
                (feat): sitemap refresh period setting + translations
              
              
                dancastellon 84fae7f
              
                (refactor): rename sitemap route handler to middleware
              
              
                dancastellon 4ea11c8
              
                (feat): recurring job that regenerates sitemaps
              
              
                dancastellon 6612677
              
                (style): corrent linting issues
              
              
                dancastellon 720b3bd
              
                (perf): limit default # URLs per sitemap to 1k + use job-collection f…
              
              
                dancastellon 28485ce
              
                (test): tests for generateSitemaps function
              
              
                dancastellon 0e56dbf
              
                (test): additional tests for generateSitemaps
              
              
                dancastellon a7b7d4a
              
                (test): app test for getSitemapXML
              
              
                dancastellon 316a6cd
              
                Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
              
              
                dancastellon 6f22ff5
              
                (style): correct import order
              
              
                dancastellon a02efcf
              
                (fix): Make settings.cart optional in CorePackageConfig schema
              
              
                dancastellon d5c641e
              
                (style): use hyphenated filenames
              
              
                dancastellon 3ad5886
              
                (fix): sitemap generation button shouldn't be main form submit
              
              
                dancastellon d2449e0
              
                (style): No need for default translation in alerts
              
              
                dancastellon 36cc784
              
                (style) no periods in translations
              
              
                dancastellon b85421f
              
                (style) methods/index.js should only import & export
              
              
                dancastellon 389bb5a
              
                (style) correct linting issues
              
              
                dancastellon 01602a2
              
                (fix): uncapitalize sitemaps.js filename
              
              
                dancastellon c6461a7
              
                (feat): only regen tag & product sitemaps when new/updated record exi…
              
              
                dancastellon 3f1fedc
              
                Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
              
              
                dancastellon 44a5992
              
                Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
              
              
                dancastellon d848c6c
              
                (refactor): max 2 params for rebuildPaginatedSitemaps function
              
              
                dancastellon 8d39151
              
                (feat): publication for last sitemap generation date
              
              
                dancastellon 734aca8
              
                (feat): better description for sitemap generation alert
              
              
                dancastellon a5e2c7b
              
                (feat): view sitemap link + last generation date in Shop Options
              
              
                dancastellon 0fba24a
              
                (style): correct eslint issues
              
              
                dancastellon 1f2164a
              
                Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
              
              
                dancastellon 7b42479
              
                (style): correct import order
              
              
                dancastellon 6c88323
              
                Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
              
              
                dancastellon 7b55ac4
              
                (fix) update sitemap generation translations
              
              
                dancastellon 20adb5c
              
                (feat): use notifications UI when manually refreshing sitemap
              
              
                dancastellon db0f013
              
                (style): correct linting issues
              
              
                dancastellon 743aea3
              
                (fix): change sitemap refresh notification text
              
              
                dancastellon a9595b4
              
                Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
              
              
                dancastellon 7abfef3
              
                (fix): provide default values for toast alerts
              
              
                dancastellon 8d5fe24
              
                (feat): sitemap refresh notification brings admin to sitemap.xml
              
              
                dancastellon 1c51a91
              
                (feat): select box for sitemap refresh period + use Reaction componen…
              
              
                dancastellon File filter
Filter by extension
Conversations
          Failed to load comments.   
        
        
          
      Loading
        
  Jump to
        
          Jump to file
        
      
      
          Failed to load files.   
        
        
          
      Loading
        
  Diff view
Diff view
There are no files selected for viewing
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
        
          
          
            59 changes: 59 additions & 0 deletions
          
          59 
        
  imports/plugins/included/sitemap-generator/client/components/sitemap-settings.js
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| import React, { Component, Fragment } from "react"; | ||
| import PropTypes from "prop-types"; | ||
| import Select from "@reactioncommerce/components/Select/v1"; | ||
| import Button from "@reactioncommerce/components/Button/v1"; | ||
| import { Translation } from "/imports/plugins/core/ui/client/components"; | ||
| 
     | 
||
| export default class SitemapSettings extends Component { | ||
| static propTypes = { | ||
| onGenerateClick: PropTypes.func.isRequired, | ||
| onRefreshPeriodChange: PropTypes.func.isRequired, | ||
| refreshPeriod: PropTypes.string.isRequired | ||
| }; | ||
| 
     | 
||
| render() { | ||
| const { onRefreshPeriodChange, onGenerateClick, refreshPeriod } = this.props; | ||
| const refreshOptions = [ | ||
| { | ||
| label: "Every 24 hours", | ||
| value: "every 24 hours" | ||
| }, | ||
| { | ||
| label: "Every 12 hours", | ||
| value: "every 12 hours" | ||
| }, | ||
| { | ||
| label: "Every hour", | ||
| value: "every 1 hour" | ||
| } | ||
| ]; | ||
| 
     | 
||
| return ( | ||
| <Fragment> | ||
| <div className="form-group" data-required="true"> | ||
| <label className="control-label" htmlFor="settings.sitemaps.refreshPeriod">Sitemap refresh period</label> | ||
| <Select | ||
| options={refreshOptions} | ||
| onChange={onRefreshPeriodChange} | ||
| value={refreshPeriod} | ||
| /> | ||
| <input | ||
| type="hidden" | ||
| name="settings.sitemaps.refreshPeriod" | ||
| data-schema-key="settings.sitemaps.refreshPeriod" | ||
| value={refreshPeriod} | ||
| /> | ||
| </div> | ||
| <div className="form-group pull-left" style={{ width: "100%", marginBottom: 30 }}> | ||
| <Button actionType="secondary" className="pull-right" isShortHeight onClick={onGenerateClick}> | ||
| <i className="rui font-icon fa fa-refresh" />  | ||
| <Translation defaultValue="Refresh sitemap" i18nKey={"shopSettings.refreshSitemapsNow"} /> | ||
| </Button> | ||
| <p className="pull-right" style={{ marginTop: 6, marginRight: 10 }}> | ||
| <a href="/sitemap.xml" target="_blank">View sitemap</a> | ||
| </p> | ||
| </div> | ||
| </Fragment> | ||
| ); | ||
| } | ||
| } | 
        
          
          
            55 changes: 55 additions & 0 deletions
          
          55 
        
  imports/plugins/included/sitemap-generator/client/containers/sitemap-settings-container.js
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| import React, { Component } from "react"; | ||
| import PropTypes from "prop-types"; | ||
| import { Meteor } from "meteor/meteor"; | ||
| import { i18next } from "/client/api"; | ||
| import SitemapSettings from "../components/sitemap-settings"; | ||
| 
     | 
||
| class SitemapSettingsContainer extends Component { | ||
| static propTypes = { | ||
| packageData: PropTypes.object | ||
| }; | ||
| 
     | 
||
| constructor(props) { | ||
| super(props); | ||
| const { packageData } = props; | ||
| const { settings } = packageData; | ||
| const { sitemaps: sitemapSettings = {} } = settings; | ||
| 
     | 
||
| this.state = { | ||
| refreshPeriod: sitemapSettings.refreshPeriod || "every 24 hours" | ||
| }; | ||
| } | ||
| 
     | 
||
| handleRefreshPeriodChange = (refreshPeriod) => { | ||
| if (refreshPeriod) { | ||
| this.setState({ refreshPeriod }); | ||
| } | ||
| }; | ||
| 
     | 
||
| handleGenerateClick = () => { | ||
| Meteor.call("sitemaps/generate", (error) => { | ||
| if (error) { | ||
| Alerts.toast(`${i18next.t("shopSettings.sitemapRefreshFailed", { | ||
| defaultValue: "Sitemap refresh failed" | ||
| })}: ${error}`, "error"); | ||
| } else { | ||
| Alerts.toast(i18next.t("shopSettings.sitemapRefreshInitiated", { | ||
| defaultValue: "Refreshing the sitemap can take up to 5 minutes. You will be notified when it is completed." | ||
| }), "success"); | ||
| } | ||
| }); | ||
| }; | ||
| 
     | 
||
| render() { | ||
| const { refreshPeriod } = this.state; | ||
| return ( | ||
| <SitemapSettings | ||
| onRefreshPeriodChange={this.handleRefreshPeriodChange} | ||
| onGenerateClick={this.handleGenerateClick} | ||
| refreshPeriod={refreshPeriod} | ||
| /> | ||
| ); | ||
| } | ||
| } | ||
| 
     | 
||
| export default SitemapSettingsContainer; | 
        
          
          
            27 changes: 27 additions & 0 deletions
          
          27 
        
  imports/plugins/included/sitemap-generator/lib/collections/schemas/sitemaps-schema.js
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| import SimpleSchema from "simpl-schema"; | ||
| import { registerSchema } from "@reactioncommerce/schemas"; | ||
| import { check } from "meteor/check"; | ||
| import { Tracker } from "meteor/tracker"; | ||
| 
     | 
||
| /** | ||
| * @name SitemapsSchema | ||
| * @memberof Schemas | ||
| * @summary Schema for Sitemaps collection | ||
| * @type {SimpleSchema} | ||
| */ | ||
| export const SitemapsSchema = new SimpleSchema({ | ||
| shopId: { | ||
| type: String | ||
| }, | ||
| handle: { | ||
| type: String | ||
| }, | ||
| xml: { | ||
| type: String | ||
| }, | ||
| createdAt: { | ||
| type: Date | ||
| } | ||
| }, { check, tracker: Tracker }); | ||
| 
     | 
||
| registerSchema("Sitemaps", SitemapsSchema); | 
        
          
          
            11 changes: 11 additions & 0 deletions
          
          11 
        
  imports/plugins/included/sitemap-generator/lib/collections/sitemaps.js
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| import { Mongo } from "meteor/mongo"; | ||
| import { SitemapsSchema } from "./schemas/sitemaps-schema"; | ||
| 
     | 
||
| /** | ||
| * @name Sitemaps | ||
| * @memberof Collections | ||
| * @summary Collection for auto-generated XML sitemaps | ||
| */ | ||
| export const Sitemaps = new Mongo.Collection("Sitemaps"); | ||
| 
     | 
||
| Sitemaps.attachSchema(SitemapsSchema); | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| import Reaction from "/imports/plugins/core/core/server/Reaction"; | ||
| 
     | 
||
| Reaction.registerPackage({ | ||
| label: "Sitemap Generator", | ||
| name: "reaction-sitemap-generator", | ||
| icon: "fa fa-vine", | ||
| autoEnable: true | ||
| }); | 
        
          
          
            13 changes: 13 additions & 0 deletions
          
          13 
        
  imports/plugins/included/sitemap-generator/server/i18n/en.json
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| [{ | ||
| "i18n": "en", | ||
| "ns": "reaction-sitemap-generator", | ||
| "translation": { | ||
| "reaction-sitemap-generator": { | ||
| "shopSettings": { | ||
| "refreshSitemapsNow": "Refresh sitemap", | ||
| "sitemapRefreshInitiated": "Refreshing the sitemap can take up to 5 minutes. You will be notified when it is completed.", | ||
| "sitemapRefreshFailed": "Sitemap refresh failed" | ||
| } | ||
| } | ||
| } | ||
| }] | 
        
          
          
            5 changes: 5 additions & 0 deletions
          
          5 
        
  imports/plugins/included/sitemap-generator/server/i18n/index.js
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| import { loadTranslations } from "/imports/plugins/core/core/server/startup/i18n"; | ||
| 
     | 
||
| import en from "./en.json"; | ||
| 
     | 
||
| loadTranslations([en]); | 
        
          
          
            21 changes: 21 additions & 0 deletions
          
          21 
        
  imports/plugins/included/sitemap-generator/server/index.js
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| import { Meteor } from "meteor/meteor"; | ||
| import { WebApp } from "meteor/webapp"; | ||
| import { Sitemaps } from "../lib/collections/sitemaps"; | ||
| import generateSitemapsJob from "./jobs/generate-sitemaps-job"; | ||
| import handleSitemapRoutes from "./middleware/handle-sitemap-routes"; | ||
| import methods from "./methods"; | ||
| 
     | 
||
| // Load translations | ||
| import "./i18n"; | ||
| 
     | 
||
| // Create a compound index to support queries by shopId or shopId & handle | ||
| Sitemaps.rawCollection().createIndex({ shopId: 1, handle: 1 }, { background: true }); | ||
| 
     | 
||
| // Setup sitemap generation recurring job | ||
| generateSitemapsJob(); | ||
| 
     | 
||
| // Sitemap front-end routes | ||
| WebApp.connectHandlers.use(handleSitemapRoutes); | ||
| 
     | 
||
| // Init methods | ||
| Meteor.methods(methods); | 
        
          
          
            63 changes: 63 additions & 0 deletions
          
          63 
        
  imports/plugins/included/sitemap-generator/server/jobs/generate-sitemaps-job.js
  
  
      
      
   
        
      
      
    
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| import Hooks from "@reactioncommerce/hooks"; | ||
| import Logger from "@reactioncommerce/logger"; | ||
| import { Jobs } from "/lib/collections"; | ||
| import Reaction from "/imports/plugins/core/core/server/Reaction"; | ||
| import { Job } from "/imports/plugins/core/job-collection/lib"; | ||
| import generateSitemaps from "../lib/generate-sitemaps"; | ||
| 
     | 
||
| /** | ||
| * @name generateSitemapsJob | ||
| * @summary Initializes and processes a job that regenerates XML sitemaps | ||
| * @returns {undefined} | ||
| */ | ||
| export default function generateSitemapsJob() { | ||
| const jobId = "sitemaps/generate"; | ||
| 
     | 
||
| // Hook that schedules job | ||
| Hooks.Events.add("afterCoreInit", () => { | ||
| const settings = Reaction.getShopSettings(); | ||
| const { sitemaps } = settings; | ||
| const refreshPeriod = (sitemaps && sitemaps.refreshPeriod) || "every 24 hours"; | ||
| 
     | 
||
| Logger.debug(`Adding ${jobId} to JobControl. Refresh ${refreshPeriod}`); | ||
| 
     | 
||
| new Job(Jobs, jobId, {}) | ||
| .retry({ | ||
| retries: 5, | ||
| wait: 60000, | ||
| backoff: "exponential" | ||
| }) | ||
| .repeat({ | ||
| schedule: Jobs.later.parse.text(refreshPeriod) | ||
| }) | ||
| .save({ | ||
| cancelRepeats: true | ||
| }); | ||
| }); | ||
| 
     | 
||
| // Function that processes job | ||
| const sitemapGenerationJob = Jobs.processJobs(jobId, { | ||
| pollInterval: 60 * 60 * 1000, // backup polling, see observer below | ||
| workTimeout: 180 * 1000 | ||
| }, (job, callback) => { | ||
| Logger.debug(`Processing ${jobId} job`); | ||
| 
     | 
||
| const { notifyUserId = "" } = job.data; | ||
| generateSitemaps({ notifyUserId }); | ||
| 
     | 
||
| const doneMessage = `${jobId} job done`; | ||
| Logger.debug(doneMessage); | ||
| job.done(doneMessage, { repeatId: true }); | ||
| callback(); | ||
| }); | ||
| 
     | 
||
| // Observer that triggers processing of job when ready | ||
| Jobs.find({ | ||
| type: jobId, | ||
| status: "ready" | ||
| }).observe({ | ||
| added() { | ||
| return sitemapGenerationJob.trigger(); | ||
| } | ||
| }); | ||
| } | 
      
      Oops, something went wrong.
        
    
  
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you explain why you needed to make this change since it doesn't appear related to this PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zenweasel Without making this field optional I'm unable to save the shop options form, unless I add an App ID for Open Exchange Rates. Shop options is where I added the sitemap refresh setting & regenerate button. I just noticed that on a fresh install, I also had to make
settings.cartin the same schema optional. Does it make sense to have the sitemap settings be in a separate form? Or perhaps somewhere else, like the Shop -> General form?