Skip to content

Commit 8bc5a4c

Browse files
committed
Merge branch release-0.16.0 into master
2 parents cefea1c + 8a9dbc5 commit 8bc5a4c

19 files changed

+647
-336
lines changed

README.md

Lines changed: 99 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
Zoom
22
====
33

4-
[![Version](https://img.shields.io/badge/version-0.15.1-5272B4.svg)](https://github.com/albrow/zoom/releases)
4+
[![Version](https://img.shields.io/badge/version-0.16.0-5272B4.svg)](https://github.com/albrow/zoom/releases)
55
[![Circle CI](https://img.shields.io/circleci/project/albrow/zoom/master.svg)](https://circleci.com/gh/albrow/zoom/tree/master)
66
[![GoDoc](https://godoc.org/github.com/albrow/zoom?status.svg)](https://godoc.org/github.com/albrow/zoom)
77

88
A blazing-fast datastore and querying engine for Go built on Redis.
99

10-
Requires Redis version >= 2.8.9 and Go version >= 1.5 with
11-
`GO15VENDOREXPERIMENT=1`. The latest version of both is recommended.
10+
Requires Redis version >= 2.8.9 and Go version >= 1.2. The latest version of
11+
both is recommended.
1212

1313
Full documentation is available on
1414
[godoc.org](http://godoc.org/github.com/albrow/zoom).
@@ -17,24 +17,45 @@ Full documentation is available on
1717
Table of Contents
1818
-----------------
1919

20+
<!-- toc -->
21+
2022
- [Development Status](#development-status)
21-
- [When is Zoom a Good Fit?](#when-is-zoom-a-good-fit)
23+
- [When is Zoom a Good Fit?](#when-is-zoom-a-good-fit-)
2224
- [Installation](#installation)
2325
- [Initialization](#initialization)
2426
- [Models](#models)
27+
* [What is a Model?](#what-is-a-model-)
28+
* [Customizing Field Names](#customizing-field-names)
29+
* [Creating Collections](#creating-collections)
30+
* [Saving Models](#saving-models)
31+
* [Updating Models](#updating-models)
32+
* [Finding a Single Model](#finding-a-single-model)
33+
* [Finding Only Certain Fields](#finding-only-certain-fields)
34+
* [Finding All Models](#finding-all-models)
35+
* [Deleting Models](#deleting-models)
36+
* [Counting the Number of Models](#counting-the-number-of-models)
2537
- [Transactions](#transactions)
2638
- [Queries](#queries)
39+
* [The Query Object](#the-query-object)
40+
* [Using Query Modifiers](#using-query-modifiers)
41+
* [A Note About String Indexes](#a-note-about-string-indexes)
2742
- [More Information](#more-information)
28-
- [Testing & Benchmarking](#testing--benchmarking)
43+
* [Persistence](#persistence)
44+
* [Atomicity](#atomicity)
45+
* [Concurrent Updates](#concurrent-updates)
46+
- [Testing & Benchmarking](#testing---benchmarking)
47+
* [Running the Tests:](#running-the-tests-)
48+
* [Running the Benchmarks:](#running-the-benchmarks-)
2949
- [Contributing](#contributing)
3050
- [Example Usage](#example-usage)
3151
- [License](#license)
3252

53+
<!-- tocstop -->
3354

3455
Development Status
3556
------------------
3657

37-
Zoom has been around for more than a year. It is well-tested and going forward the API
58+
Zoom was first started in 2013. It is well-tested and going forward the API
3859
will be relatively stable. We are closing in on Version 1.0.0-alpha.
3960

4061
At this time, Zoom can be considered safe for use in low-traffic production
@@ -207,14 +228,16 @@ type Person struct {
207228
Because of the way Zoom uses reflection, all the fields you want to save need to be exported.
208229
Unexported fields (including unexported embedded structs with exported fields) will not
209230
be saved. This is a departure from how the encoding/json and encoding/xml packages
210-
behave. See [issue #25](https://github.com/albrow/zoom/issues/25) for discussion. Almost
211-
any type of field is supported, including custom types, slices, maps, complex types, and embedded
212-
structs. The only things that are not supported are recursive data structures and functions.
231+
behave. See [issue #25](https://github.com/albrow/zoom/issues/25) for discussion.
232+
233+
Almost any type of field is supported, including custom types, slices, maps, complex types,
234+
and embedded structs. The only things that are not supported are recursive data structures and
235+
functions.
213236

214237
### Customizing Field Names
215238

216239
You can change the name used to store the field in Redis with the `redis:"<name>"` struct tag. So
217-
for example, if you wanted the fields to be stored as lowercase fields in redis, you could use the
240+
for example, if you wanted the fields to be stored as lowercase fields in Redis, you could use the
218241
following struct definition:
219242

220243
``` go
@@ -262,13 +285,13 @@ type CollectionOptions struct {
262285
// provides JSONMarshalerUnmarshaler to support json encoding out of the box.
263286
// Default: GobMarshalerUnmarshaler.
264287
FallbackMarshalerUnmarshaler MarshalerUnmarshaler
265-
// Iff Index is true, any model in the collection that is saved will be added
266-
// to a set in redis which acts as an index. The default value is false. The
288+
// If Index is true, any model in the collection that is saved will be added
289+
// to a set in Redis which acts as an index. The default value is false. The
267290
// key for the set is exposed via the IndexKey method. Queries and the
268291
// FindAll, Count, and DeleteAll methods will not work for unindexed
269292
// collections. This may change in future versions. Default: false.
270293
Index bool
271-
// Name is a unique string identifier to use for the collection in redis. All
294+
// Name is a unique string identifier to use for the collection in Redis. All
272295
// models in this collection that are saved in the database will use the
273296
// collection name as a prefix. If not provided, the default name will be the
274297
// name of the model type without the package prefix or pointer declarations.
@@ -458,24 +481,30 @@ t := pool.NewTransaction()
458481
t.Save(People, &Person{Name: "Foo"})
459482
t.Save(People, &Person{Name: "Bar"})
460483
// Count expects a pointer to an integer, which it will change the value of
461-
// when the transaction is executed. If you don't care about the number of
462-
// models deleted, you can pass in nil.
484+
// when the transaction is executed.
463485
t.Count(People, &numPeople)
464486
if err := t.Exec(); err != nil {
465487
// handle error
466488
}
467-
// numPeople will now equal the number of *Person models in the database
489+
// numPeople will now equal the number of `Person` models in the database
468490
fmt.Println(numPeople)
469491
// Output:
470492
// 2
471493
```
472494

473-
You can also execute custom Redis commands or run lua scripts with the
495+
You can execute custom Redis commands or run custom Lua scripts inside a
496+
[`Transaction`](http://godoc.org/github.com/albrow/zoom/#Transaction) using the
474497
[`Command`](http://godoc.org/github.com/albrow/zoom/#Transaction.Command) and
475-
[`Script`](http://godoc.org/github.com/albrow/zoom/#Transaction.Script) methods. Both methods expect a
476-
[`ReplyHandler`](http://godoc.org/github.com/albrow/zoom/#ReplyHandler) as an argument. A `ReplyHandler` is
477-
simply a function that will do something with the reply from Redis corresponding to the script or command
478-
that was run. `ReplyHandler`'s are executed in order when you call `Exec`.
498+
[`Script`](http://godoc.org/github.com/albrow/zoom/#Transaction.Script) methods.
499+
Both methods expect a
500+
[`ReplyHandler`](http://godoc.org/github.com/albrow/zoom/#ReplyHandler) as an
501+
argument. A `ReplyHandler` is simply a function that will do something with the
502+
reply from Redis. `ReplyHandler`'s are executed in order when you call `Exec`.
503+
504+
Right out of the box, Zoom exports a few useful `ReplyHandler`s. These include
505+
handlers for the primitive types `int`, `string`, `bool`, and `float64`, as well
506+
as handlers for scanning a reply into a `Model` or a slice of `Model`s. You can
507+
also write your own custom `ReplyHandler`s if needed.
479508

480509

481510
Queries
@@ -546,10 +575,10 @@ More Information
546575
### Persistence
547576

548577
Zoom is as persistent as the underlying Redis database. If you intend to use Redis as a permanent
549-
datastore, it is recommended that you turn on both AOF and RDB persistence options and set fsync to
550-
everysec. This will give you good performance while making data loss highly unlikely.
578+
datastore, it is recommended that you turn on both AOF and RDB persistence options and set `fsync` to
579+
`everysec`. This will give you good performance while making data loss highly unlikely.
551580

552-
If you want greater protections against data loss, you can set fsync to always. This will hinder performance
581+
If you want greater protections against data loss, you can set `fsync` to `always`. This will hinder performance
553582
but give you persistence guarantees
554583
[very similar to SQL databases such as PostgreSQL](http://redis.io/topics/persistence#ok-so-what-should-i-use).
555584

@@ -558,13 +587,13 @@ but give you persistence guarantees
558587
### Atomicity
559588

560589
All methods and functions in Zoom that touch the database do so atomically. This is accomplished using
561-
Redis transactions and lua scripts when necessary. What this means is that Zoom will not
590+
Redis transactions and Lua scripts when necessary. What this means is that Zoom will not
562591
put Redis into an inconsistent state (e.g. where indexes to not match the rest of the data).
563592

564593
However, it should be noted that there is a caveat with Redis atomicity guarantees. If Redis crashes
565594
in the middle of a transaction or script execution, it is possible that your AOF file can become
566595
corrupted. If this happens, Redis will refuse to start until the AOF file is fixed. It is relatively
567-
easy to fix the problem with the redis-check-aof tool, which will remove the partial transaction
596+
easy to fix the problem with the `redis-check-aof` tool, which will remove the partial transaction
568597
from the AOF file.
569598

570599
If you intend to issue custom Redis commands or run custom scripts, it is highly recommended that
@@ -582,12 +611,13 @@ Read more about:
582611

583612
### Concurrent Updates
584613

585-
Currently, Zoom does not support concurrent "read before write" updates on
586-
models. The `UpdateFields` method introduced in version 0.12 offers some
614+
Currently, Zoom does not directly support concurrent "read before write" updates
615+
on models. The `UpdateFields` method introduced in version 0.12 offers some
587616
additional safety for concurrent updates, as long as no concurrent callers
588617
update the same fields (or if you are okay with updates overwriting previous
589618
changes). However, cases where you need to do a "read before write" update are
590-
still not safe by default. For example, consider the following code:
619+
still not safe if you use a naive implementation. For example, consider the
620+
following code:
591621

592622
``` go
593623
func likePost(postId string) error {
@@ -609,38 +639,52 @@ The line `post.Likes += 1` is a "read before write" operation. That's because
609639
the `+=` operator implicitly reads the current value of `post.Likes` and then
610640
adds to it.
611641

612-
This can cause a bug if the function is called across multiple threads or
642+
This can cause a bug if the function is called across multiple goroutines or
613643
multiple machines concurrently, because the `Post` model can change in between
614644
the time we retrieved it from the database with `Find` and saved it again with
615-
`Save`. Future versions of Zoom may provide
616-
[optimistic locking](https://github.com/albrow/zoom/issues/13) or other means to
617-
avoid these kinds of errors. In the meantime, you could fix this code by using
618-
an `HINCRBY` command directly like so:
645+
`Save`.
619646

620-
``` go
621-
func likePost(postId string) error {
622-
// modelKey is the key of the main hash for the model, which
623-
// stores the struct fields as hash fields in Redis.
624-
modelKey, err := Posts.ModelKey(postId)
625-
if err != nil {
626-
return err
627-
}
628-
conn := zoom.NewConn()
629-
defer conn.Close()
630-
if _, err := conn.Do("HINCRBY", modelKey, 1); err != nil {
631-
return err
632-
}
647+
However, since Zoom allows you to run your own Redis commands, you could fix
648+
this code by manually using HINCRBY:
649+
650+
```go
651+
// likePost atomically increments the number of likes for a post with the given
652+
// id and then returns the new number of likes.
653+
func likePost(postId string) (int, error) {
654+
// Get the key which is used to store the post in Redis
655+
postKey := Posts.ModelKey(postId)
656+
// Start a new transaction
657+
tx := pool.NewTransaction()
658+
// Add a command to increment the number of Likes. The HINCRBY command returns
659+
// an integer which we will scan into numLikes.
660+
var numLikes int
661+
tx.Command(
662+
"HINCRBY",
663+
redis.Args{postKey, "Likes", 1},
664+
zoom.NewScanIntHandler(&numLikes),
665+
)
666+
if err := tx.Exec(); err != nil {
667+
return 0, err
668+
}
669+
return numLikes, nil
633670
}
634671
```
635672

636-
You could also use a lua script, which have full transactional support in Zoom,
637-
for more complicated "read before write" updates.
673+
Future versions of Zoom may provide
674+
[optimistic locking](https://github.com/albrow/zoom/issues/13) or other means to
675+
make "read before write" updates easier.
676+
677+
Read more about:
678+
- [Redis Commands](http://redis.io/commands)
679+
- [Redigo](https://github.com/garyburd/redigo), the Redis Driver used by Zoom
680+
- [`ReplyHandler`s provided by Zoom](https://godoc.org/github.com/albrow/zoom)
681+
- [How Zoom works Under the Hood](https://github.com/albrow/zoom/wiki/Under-the-Hood)
638682

639683

640684
Testing & Benchmarking
641685
----------------------
642686

643-
### Running the Tests:
687+
### Running the Tests
644688

645689
To run the tests, make sure you're in the root directory for Zoom and run:
646690

@@ -665,7 +709,7 @@ you could use:
665709
go test -network=unix -address=/tmp/redis.sock -database=3
666710
```
667711

668-
### Running the Benchmarks:
712+
### Running the Benchmarks
669713

670714
To run the benchmarks, make sure you're in the root directory for the project and run:
671715

@@ -737,10 +781,10 @@ See [CONTRIBUTING.md](https://github.com/albrow/zoom/blob/master/CONTRIBUTING.md
737781
Example Usage
738782
-------------
739783

740-
There is an [example json/rest application](https://github.com/albrow/peeps-negroni)
741-
which uses the latest version of Zoom. It is a simple example that doesn't use all of
742-
Zoom's features, but should be good enough for understanding how zoom can work in a
743-
real application.
784+
[albrow/people](https://github.com/albrow/people) is an example HTTP/JSON API
785+
which uses the latest version of Zoom. It is a simple example that doesn't use
786+
all of Zoom's features, but should be good enough for understanding how Zoom can
787+
work in a real application.
744788

745789

746790
License

collection.go

Lines changed: 32 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -137,9 +137,16 @@ func (p *Pool) nameIsRegistered(name string) bool {
137137

138138
// ModelKey returns the key that identifies a hash in the database
139139
// which contains all the fields of the model corresponding to the given
140-
// id. It returns an error iff id is empty.
141-
func (c *Collection) ModelKey(id string) (string, error) {
142-
return c.spec.modelKey(id)
140+
// id. If id is an empty string, it will return an empty string.
141+
func (c *Collection) ModelKey(id string) string {
142+
if id == "" {
143+
return ""
144+
}
145+
// c.spec.modelKey(id) will only return an error if id was an empty string.
146+
// Since we already ruled that out with the check above, we can safely ignore
147+
// the error return value here.
148+
key, _ := c.spec.modelKey(id)
149+
return key
143150
}
144151

145152
// IndexKey returns the key that identifies a set in the database that
@@ -155,6 +162,22 @@ func (c *Collection) FieldIndexKey(fieldName string) (string, error) {
155162
return c.spec.fieldIndexKey(fieldName)
156163
}
157164

165+
// FieldNames returns all the field names for the Collection. The order is
166+
// always the same and is used internally by Zoom to determine the order of
167+
// fields in Redis commands such as HMGET.
168+
func (c *Collection) FieldNames() []string {
169+
return c.spec.fieldNames()
170+
}
171+
172+
// FieldRedisNames returns all the Redis names for the fields of the Collection.
173+
// For example, if a Collection was created with a model type that includes
174+
// custom field names via the `redis` struct tag, those names will be returned.
175+
// The order is always the same and is used internally by Zoom to determine the
176+
// order of fields in Redis commands such as HMGET.
177+
func (c *Collection) FieldRedisNames() []string {
178+
return c.spec.fieldRedisNames()
179+
}
180+
158181
// newNilCollectionError returns an error with a message describing that
159182
// methodName was called on a nil collection.
160183
func newNilCollectionError(methodName string) error {
@@ -405,7 +428,7 @@ func (t *Transaction) Find(c *Collection, id string, model Model) {
405428
for _, fieldName := range mr.spec.fieldRedisNames() {
406429
args = append(args, fieldName)
407430
}
408-
t.Command("HMGET", args, newScanModelHandler(mr.spec.fieldNames(), mr))
431+
t.Command("HMGET", args, newScanModelRefHandler(mr.spec.fieldNames(), mr))
409432
}
410433

411434
// FindFields is like Find but finds and sets only the specified fields. Any
@@ -450,7 +473,7 @@ func (t *Transaction) FindFields(c *Collection, id string, fieldNames []string,
450473
args = append(args, c.spec.fieldsByName[fieldName].redisName)
451474
}
452475
// Get the fields from the main hash for this model
453-
t.Command("HMGET", args, newScanModelHandler(fieldNames, mr))
476+
t.Command("HMGET", args, newScanModelRefHandler(fieldNames, mr))
454477
}
455478

456479
// FindAll finds all the models of the given type. It executes the commands needed
@@ -523,7 +546,7 @@ func (t *Transaction) Count(c *Collection, count *int) {
523546
t.setError(newUnindexedCollectionError("Count"))
524547
return
525548
}
526-
t.Command("SCARD", redis.Args{c.IndexKey()}, newScanIntHandler(count))
549+
t.Command("SCARD", redis.Args{c.IndexKey()}, NewScanIntHandler(count))
527550
}
528551

529552
// Delete removes the model with the given type and id from the database. It will
@@ -561,7 +584,7 @@ func (t *Transaction) Delete(c *Collection, id string, deleted *bool) {
561584
if deleted == nil {
562585
handler = nil
563586
} else {
564-
handler = newScanBoolHandler(deleted)
587+
handler = NewScanBoolHandler(deleted)
565588
}
566589
// Delete the main hash
567590
t.Command("DEL", redis.Args{c.Name() + ":" + id}, handler)
@@ -626,9 +649,9 @@ func (t *Transaction) DeleteAll(c *Collection, count *int) {
626649
if count == nil {
627650
handler = nil
628651
} else {
629-
handler = newScanIntHandler(count)
652+
handler = NewScanIntHandler(count)
630653
}
631-
t.deleteModelsBySetIds(c.IndexKey(), c.Name(), handler)
654+
t.DeleteModelsBySetIds(c.IndexKey(), c.Name(), handler)
632655
}
633656

634657
// checkModelType returns an error iff model is not of the registered type that

collection_test.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@ func TestSave(t *testing.T) {
129129

130130
// Make sure the model was saved correctly
131131
expectModelExists(t, testModels, model)
132-
key, _ := testModels.ModelKey(model.ModelId())
132+
key := testModels.ModelKey(model.ModelId())
133133
mu := testModels.spec.fallback
134134
expectFieldEquals(t, key, "Int", mu, model.Int)
135135
expectFieldEquals(t, key, "String", mu, model.String)
@@ -158,7 +158,7 @@ func TestUpdateFields(t *testing.T) {
158158

159159
// Make sure the model was saved correctly
160160
expectModelExists(t, testModels, model)
161-
key, _ := testModels.ModelKey(model.ModelId())
161+
key := testModels.ModelKey(model.ModelId())
162162
mu := testModels.spec.fallback
163163
expectFieldEquals(t, key, "Int", mu, model.Int)
164164
expectFieldEquals(t, key, "String", mu, originalString)

0 commit comments

Comments
 (0)