-
Notifications
You must be signed in to change notification settings - Fork 2.2k
#4353 Plugin for auto-generated sitemaps #4413
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
aldeed
merged 43 commits into
release-1.15.0
from
feat-4353-dancastellon-sitemap-generator
Aug 16, 2018
Merged
Changes from all commits
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
c7a0d51
(feat): generate sitemaps on demand and store in collection
dancastellon f671b6c
(style): correct linting issues
dancastellon 59226ad
(feat): Routes for auto-generated sitemap files
dancastellon db2d035
(feat): Add compound index for Sitemaps queries
dancastellon 56de04e
(refactor): Cleanup generateSitemaps function
dancastellon b423901
(style): corrent linting issues
dancastellon a7a11c2
(feat): sitemap refresh period setting + translations
dancastellon 84fae7f
(refactor): rename sitemap route handler to middleware
dancastellon 4ea11c8
(feat): recurring job that regenerates sitemaps
dancastellon 6612677
(style): corrent linting issues
dancastellon 720b3bd
(perf): limit default # URLs per sitemap to 1k + use job-collection f…
dancastellon 28485ce
(test): tests for generateSitemaps function
dancastellon 0e56dbf
(test): additional tests for generateSitemaps
dancastellon a7b7d4a
(test): app test for getSitemapXML
dancastellon 316a6cd
Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
dancastellon 6f22ff5
(style): correct import order
dancastellon a02efcf
(fix): Make settings.cart optional in CorePackageConfig schema
dancastellon d5c641e
(style): use hyphenated filenames
dancastellon 3ad5886
(fix): sitemap generation button shouldn't be main form submit
dancastellon d2449e0
(style): No need for default translation in alerts
dancastellon 36cc784
(style) no periods in translations
dancastellon b85421f
(style) methods/index.js should only import & export
dancastellon 389bb5a
(style) correct linting issues
dancastellon 01602a2
(fix): uncapitalize sitemaps.js filename
dancastellon c6461a7
(feat): only regen tag & product sitemaps when new/updated record exi…
dancastellon 3f1fedc
Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
dancastellon 44a5992
Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
dancastellon d848c6c
(refactor): max 2 params for rebuildPaginatedSitemaps function
dancastellon 8d39151
(feat): publication for last sitemap generation date
dancastellon 734aca8
(feat): better description for sitemap generation alert
dancastellon a5e2c7b
(feat): view sitemap link + last generation date in Shop Options
dancastellon 0fba24a
(style): correct eslint issues
dancastellon 1f2164a
Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
dancastellon 7b42479
(style): correct import order
dancastellon 6c88323
Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
dancastellon 7b55ac4
(fix) update sitemap generation translations
dancastellon 20adb5c
(feat): use notifications UI when manually refreshing sitemap
dancastellon db0f013
(style): correct linting issues
dancastellon 743aea3
(fix): change sitemap refresh notification text
dancastellon a9595b4
Merge branch 'release-1.14.0' of https://github.com/reactioncommerce/…
dancastellon 7abfef3
(fix): provide default values for toast alerts
dancastellon 8d5fe24
(feat): sitemap refresh notification brings admin to sitemap.xml
dancastellon 1c51a91
(feat): select box for sitemap refresh period + use Reaction componen…
dancastellon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
59 changes: 59 additions & 0 deletions
59
imports/plugins/included/sitemap-generator/client/components/sitemap-settings.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| import React, { Component, Fragment } from "react"; | ||
| import PropTypes from "prop-types"; | ||
| import Select from "@reactioncommerce/components/Select/v1"; | ||
| import Button from "@reactioncommerce/components/Button/v1"; | ||
| import { Translation } from "/imports/plugins/core/ui/client/components"; | ||
|
|
||
| export default class SitemapSettings extends Component { | ||
| static propTypes = { | ||
| onGenerateClick: PropTypes.func.isRequired, | ||
| onRefreshPeriodChange: PropTypes.func.isRequired, | ||
| refreshPeriod: PropTypes.string.isRequired | ||
| }; | ||
|
|
||
| render() { | ||
| const { onRefreshPeriodChange, onGenerateClick, refreshPeriod } = this.props; | ||
| const refreshOptions = [ | ||
| { | ||
| label: "Every 24 hours", | ||
| value: "every 24 hours" | ||
| }, | ||
| { | ||
| label: "Every 12 hours", | ||
| value: "every 12 hours" | ||
| }, | ||
| { | ||
| label: "Every hour", | ||
| value: "every 1 hour" | ||
| } | ||
| ]; | ||
|
|
||
| return ( | ||
| <Fragment> | ||
| <div className="form-group" data-required="true"> | ||
| <label className="control-label" htmlFor="settings.sitemaps.refreshPeriod">Sitemap refresh period</label> | ||
| <Select | ||
| options={refreshOptions} | ||
| onChange={onRefreshPeriodChange} | ||
| value={refreshPeriod} | ||
| /> | ||
| <input | ||
| type="hidden" | ||
| name="settings.sitemaps.refreshPeriod" | ||
| data-schema-key="settings.sitemaps.refreshPeriod" | ||
| value={refreshPeriod} | ||
| /> | ||
| </div> | ||
| <div className="form-group pull-left" style={{ width: "100%", marginBottom: 30 }}> | ||
| <Button actionType="secondary" className="pull-right" isShortHeight onClick={onGenerateClick}> | ||
| <i className="rui font-icon fa fa-refresh" /> | ||
| <Translation defaultValue="Refresh sitemap" i18nKey={"shopSettings.refreshSitemapsNow"} /> | ||
| </Button> | ||
| <p className="pull-right" style={{ marginTop: 6, marginRight: 10 }}> | ||
| <a href="/sitemap.xml" target="_blank">View sitemap</a> | ||
| </p> | ||
| </div> | ||
| </Fragment> | ||
| ); | ||
| } | ||
| } |
55 changes: 55 additions & 0 deletions
55
imports/plugins/included/sitemap-generator/client/containers/sitemap-settings-container.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| import React, { Component } from "react"; | ||
| import PropTypes from "prop-types"; | ||
| import { Meteor } from "meteor/meteor"; | ||
| import { i18next } from "/client/api"; | ||
| import SitemapSettings from "../components/sitemap-settings"; | ||
|
|
||
| class SitemapSettingsContainer extends Component { | ||
| static propTypes = { | ||
| packageData: PropTypes.object | ||
| }; | ||
|
|
||
| constructor(props) { | ||
| super(props); | ||
| const { packageData } = props; | ||
| const { settings } = packageData; | ||
| const { sitemaps: sitemapSettings = {} } = settings; | ||
|
|
||
| this.state = { | ||
| refreshPeriod: sitemapSettings.refreshPeriod || "every 24 hours" | ||
| }; | ||
| } | ||
|
|
||
| handleRefreshPeriodChange = (refreshPeriod) => { | ||
| if (refreshPeriod) { | ||
| this.setState({ refreshPeriod }); | ||
| } | ||
| }; | ||
|
|
||
| handleGenerateClick = () => { | ||
| Meteor.call("sitemaps/generate", (error) => { | ||
| if (error) { | ||
| Alerts.toast(`${i18next.t("shopSettings.sitemapRefreshFailed", { | ||
| defaultValue: "Sitemap refresh failed" | ||
| })}: ${error}`, "error"); | ||
| } else { | ||
| Alerts.toast(i18next.t("shopSettings.sitemapRefreshInitiated", { | ||
| defaultValue: "Refreshing the sitemap can take up to 5 minutes. You will be notified when it is completed." | ||
| }), "success"); | ||
| } | ||
| }); | ||
| }; | ||
|
|
||
| render() { | ||
| const { refreshPeriod } = this.state; | ||
| return ( | ||
| <SitemapSettings | ||
| onRefreshPeriodChange={this.handleRefreshPeriodChange} | ||
| onGenerateClick={this.handleGenerateClick} | ||
| refreshPeriod={refreshPeriod} | ||
| /> | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| export default SitemapSettingsContainer; |
27 changes: 27 additions & 0 deletions
27
imports/plugins/included/sitemap-generator/lib/collections/schemas/sitemaps-schema.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| import SimpleSchema from "simpl-schema"; | ||
| import { registerSchema } from "@reactioncommerce/schemas"; | ||
| import { check } from "meteor/check"; | ||
| import { Tracker } from "meteor/tracker"; | ||
|
|
||
| /** | ||
| * @name SitemapsSchema | ||
| * @memberof Schemas | ||
| * @summary Schema for Sitemaps collection | ||
| * @type {SimpleSchema} | ||
| */ | ||
| export const SitemapsSchema = new SimpleSchema({ | ||
| shopId: { | ||
| type: String | ||
| }, | ||
| handle: { | ||
| type: String | ||
| }, | ||
| xml: { | ||
| type: String | ||
| }, | ||
| createdAt: { | ||
| type: Date | ||
| } | ||
| }, { check, tracker: Tracker }); | ||
|
|
||
| registerSchema("Sitemaps", SitemapsSchema); |
11 changes: 11 additions & 0 deletions
11
imports/plugins/included/sitemap-generator/lib/collections/sitemaps.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| import { Mongo } from "meteor/mongo"; | ||
| import { SitemapsSchema } from "./schemas/sitemaps-schema"; | ||
|
|
||
| /** | ||
| * @name Sitemaps | ||
| * @memberof Collections | ||
| * @summary Collection for auto-generated XML sitemaps | ||
| */ | ||
| export const Sitemaps = new Mongo.Collection("Sitemaps"); | ||
|
|
||
| Sitemaps.attachSchema(SitemapsSchema); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| import Reaction from "/imports/plugins/core/core/server/Reaction"; | ||
|
|
||
| Reaction.registerPackage({ | ||
| label: "Sitemap Generator", | ||
| name: "reaction-sitemap-generator", | ||
| icon: "fa fa-vine", | ||
| autoEnable: true | ||
| }); |
13 changes: 13 additions & 0 deletions
13
imports/plugins/included/sitemap-generator/server/i18n/en.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| [{ | ||
| "i18n": "en", | ||
| "ns": "reaction-sitemap-generator", | ||
| "translation": { | ||
| "reaction-sitemap-generator": { | ||
| "shopSettings": { | ||
| "refreshSitemapsNow": "Refresh sitemap", | ||
| "sitemapRefreshInitiated": "Refreshing the sitemap can take up to 5 minutes. You will be notified when it is completed.", | ||
| "sitemapRefreshFailed": "Sitemap refresh failed" | ||
| } | ||
| } | ||
| } | ||
| }] |
5 changes: 5 additions & 0 deletions
5
imports/plugins/included/sitemap-generator/server/i18n/index.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| import { loadTranslations } from "/imports/plugins/core/core/server/startup/i18n"; | ||
|
|
||
| import en from "./en.json"; | ||
|
|
||
| loadTranslations([en]); |
21 changes: 21 additions & 0 deletions
21
imports/plugins/included/sitemap-generator/server/index.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| import { Meteor } from "meteor/meteor"; | ||
| import { WebApp } from "meteor/webapp"; | ||
| import { Sitemaps } from "../lib/collections/sitemaps"; | ||
| import generateSitemapsJob from "./jobs/generate-sitemaps-job"; | ||
| import handleSitemapRoutes from "./middleware/handle-sitemap-routes"; | ||
| import methods from "./methods"; | ||
|
|
||
| // Load translations | ||
| import "./i18n"; | ||
|
|
||
| // Create a compound index to support queries by shopId or shopId & handle | ||
| Sitemaps.rawCollection().createIndex({ shopId: 1, handle: 1 }, { background: true }); | ||
|
|
||
| // Setup sitemap generation recurring job | ||
| generateSitemapsJob(); | ||
|
|
||
| // Sitemap front-end routes | ||
| WebApp.connectHandlers.use(handleSitemapRoutes); | ||
|
|
||
| // Init methods | ||
| Meteor.methods(methods); |
63 changes: 63 additions & 0 deletions
63
imports/plugins/included/sitemap-generator/server/jobs/generate-sitemaps-job.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| import Hooks from "@reactioncommerce/hooks"; | ||
| import Logger from "@reactioncommerce/logger"; | ||
| import { Jobs } from "/lib/collections"; | ||
| import Reaction from "/imports/plugins/core/core/server/Reaction"; | ||
| import { Job } from "/imports/plugins/core/job-collection/lib"; | ||
| import generateSitemaps from "../lib/generate-sitemaps"; | ||
|
|
||
| /** | ||
| * @name generateSitemapsJob | ||
| * @summary Initializes and processes a job that regenerates XML sitemaps | ||
| * @returns {undefined} | ||
| */ | ||
| export default function generateSitemapsJob() { | ||
| const jobId = "sitemaps/generate"; | ||
|
|
||
| // Hook that schedules job | ||
| Hooks.Events.add("afterCoreInit", () => { | ||
| const settings = Reaction.getShopSettings(); | ||
| const { sitemaps } = settings; | ||
| const refreshPeriod = (sitemaps && sitemaps.refreshPeriod) || "every 24 hours"; | ||
|
|
||
| Logger.debug(`Adding ${jobId} to JobControl. Refresh ${refreshPeriod}`); | ||
|
|
||
| new Job(Jobs, jobId, {}) | ||
| .retry({ | ||
| retries: 5, | ||
| wait: 60000, | ||
| backoff: "exponential" | ||
| }) | ||
| .repeat({ | ||
| schedule: Jobs.later.parse.text(refreshPeriod) | ||
| }) | ||
| .save({ | ||
| cancelRepeats: true | ||
| }); | ||
| }); | ||
|
|
||
| // Function that processes job | ||
| const sitemapGenerationJob = Jobs.processJobs(jobId, { | ||
| pollInterval: 60 * 60 * 1000, // backup polling, see observer below | ||
| workTimeout: 180 * 1000 | ||
| }, (job, callback) => { | ||
| Logger.debug(`Processing ${jobId} job`); | ||
|
|
||
| const { notifyUserId = "" } = job.data; | ||
| generateSitemaps({ notifyUserId }); | ||
|
|
||
| const doneMessage = `${jobId} job done`; | ||
| Logger.debug(doneMessage); | ||
| job.done(doneMessage, { repeatId: true }); | ||
| callback(); | ||
| }); | ||
|
|
||
| // Observer that triggers processing of job when ready | ||
| Jobs.find({ | ||
| type: jobId, | ||
| status: "ready" | ||
| }).observe({ | ||
| added() { | ||
| return sitemapGenerationJob.trigger(); | ||
| } | ||
| }); | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you explain why you needed to make this change since it doesn't appear related to this PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zenweasel Without making this field optional I'm unable to save the shop options form, unless I add an App ID for Open Exchange Rates. Shop options is where I added the sitemap refresh setting & regenerate button. I just noticed that on a fresh install, I also had to make
settings.cartin the same schema optional. Does it make sense to have the sitemap settings be in a separate form? Or perhaps somewhere else, like the Shop -> General form?