Manage Go dependencies using dep


Update @ 2018-11-26: Technology is not just moving at a breakneck speed but also changing rapidly. Within a year, this article is OUTDATED!

And according to the dep project page:

dep was the “official experiment.” The Go toolchain, as of 1.11, has (experimentally) adopted an approach that sharply diverges from dep. As a result, we are continuing development of dep, but gearing work primarily towards the development of an alternative prototype for versioning behavior in the toolchain.

For more information about the new Go build-in management, please refer to the official GitHub Wiki - Go 1.11 Modules.

Thanks John Arundel @bitfield and Erhan Yakut @yakuter for revealing the problem. 🙇


Update @ 2018-02-03: Sam Boyer from the godep team has clarified some incorrect information in this article. I apologize to Sam Boyer and the readers for any inconvenience. 😢


Previously i posted an article about dependency management in Go using Glide and i got a feedback about Glide will become obsolete and the Glide team is suggesting users moving to a another dependency management tool called dep written by the golang team.

The Go community now has the dep project to manage dependencies. Please consider trying to migrate from Glide to dep… Glide will continue to be supported for some time but is considered to be in a state of support rather than active feature development.

There is a plan about integrating dep into the toolchain in Go 1.10 release but seems it still has a way to go.

Update @ 2018-02-03:

  • dep is officially released.
  • dep is not moving into the toolchain with 1.10. please refer to the roadmap for the latest information.
And i am just not fast enough. 🐌

Create the project inside $GOPATH

Same as before, let’s create a new project with root path $GOPATH/src/gitlab.com/ykyuen/dep-example and add the following file.

main.go

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
package main

import humanize "github.com/dustin/go-humanize"
import "fmt"

func main() {
  fmt.Println("hello world")
  fmt.Printf("That file is %s.\n", humanize.Bytes(82854982)) // That file is 83 MB.
  fmt.Printf("You're my %s best friend.\n", humanize.Ordinal(193)) // You are my 193rd best friend.
  fmt.Printf("You owe $%s.\n", humanize.Comma(6582491)) // You owe $6,582,491.
}

The dep way

Gopkg.toml and Gopkg.lock

dep reads two files called Gopkg.toml and the Gopkg.lock. Let’s initialize these 2 files.

1
2
3
[[email protected] dep-example]$ dep init
  Using master as constraint for direct dep github.com/dustin/go-humanize
  Locking in master (bb3d318) for direct dep github.com/dustin/go-humanize

As you can see, the dep init command scans the source codes and downloads all the packages needed for the project into the vendor folder.

The Gopkg.lock serves exactly the same function as the glide.lock file which locks the version of the packages EXCEPT the version should be maintained in the Gopkg.toml. In short, the Gopkg.lock file is auto-generated and it depends on the import statements in the source with version controlled by Gopkg.toml.

Update dependency’s version

Let’s edit the Gopkg.toml and use a slightly older version of the go-humanize package instead of the latest master branch.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# Gopkg.toml example
#
# Refer to https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
#   name = "github.com/user/project"
#   version = "1.0.0"
#
# [[constraint]]
#   name = "github.com/user/project2"
#   branch = "dev"
#   source = "github.com/myfork/project2"
#
# [[override]]
#  name = "github.com/x/y"
#  version = "2.4.0"


[[constraint]]
  #branch = "master"
  name = "github.com/dustin/go-humanize"
  revision = "0b19b17f90333e44518aa31bbf8126017960aee3"

Then run dep ensure to update the package to the desired version. The following is the diff of the updated Gopkg.lock.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
--- /home/ykyuen/go/src/gitlab.com/ykyuen/dep-example/Gopkg.lock
+++ # This file is autogenerated, do not edit; changes
@@ -2,14 +2,13 @@
 
 
 [[projects]]
-  branch = "master"
   name = "github.com/dustin/go-humanize"
   packages = ["."]
-  revision = "bb3d318650d48840a39aa21a027c6630e198e626"
+  revision = "0b19b17f90333e44518aa31bbf8126017960aee3"
 
 [solve-meta]
   analyzer-name = "dep"
   analyzer-version = 1
-  inputs-digest = "a5d65a2bdf47e41b99ad71813142ae93970f5806c77630b2aea665fe631dde23"
+  inputs-digest = "0023bfe634a061b89ae0fbd71e3236f3f75f0843a0c974eeb9822040c0ea2dc4"
   solver-name = "gps-cdcl"
   solver-version = 1
 

Add a new dependency

New package could be added using the dep ensure -add command.

1
2
3
4
5
[[email protected] dep-example]$ dep ensure -add github.com/leekchan/accounting
Fetching sources...

"github.com/leekchan/accounting" is not imported by your project, and has been temporarily added to Gopkg.lock and vendor/.
If you run "dep ensure" again before actually importing it, it will disappear from Gopkg.lock and vendor/.

Now we have the new accounting package ready in the vendor folder with new constraints written to Gopkg.toml and locked in Gopkg.lock. Let’s update the main.go as follow.

main.go

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
package main

import humanize "github.com/dustin/go-humanize"
import accounting "github.com/leekchan/accounting"
import "math/big"
import "fmt"

func main() {
  fmt.Println("hello world")
  fmt.Printf("That file is %s.\n", humanize.Bytes(82854982)) // That file is 83 MB.
  fmt.Printf("You're my %s best friend.\n", humanize.Ordinal(193)) // You are my 193rd best friend.
  fmt.Printf("You owe $%s.\n", humanize.Comma(6582491)) // You owe $6,582,491.

  ac := accounting.Accounting{Symbol: "$", Precision: 2}
  fmt.Println(ac.FormatMoney(123456789.213123))                       // "$123,456,789.21"
  fmt.Println(ac.FormatMoney(12345678))                               // "$12,345,678.00"
  fmt.Println(ac.FormatMoney(big.NewRat(77777777, 3)))                // "$25,925,925.67"
  fmt.Println(ac.FormatMoney(big.NewRat(-77777777, 3)))               // "-$25,925,925.67"
  fmt.Println(ac.FormatMoneyBigFloat(big.NewFloat(123456789.213123))) // "$123,456,789.21"
}

And run it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
[[email protected] dep-example]$ go run main.go
hello world
That file is 83 MB.
You're my 193rd best friend.
You owe $6,582,491.
$123,456,789.21
$12,345,678.00
$25,925,925.67
-$25,925,925.67
$123,456,789.21

The issue with git submodule

One major difference of dep compared to Glide is the package’s submodule is ignored. For example, after adding the go-goracle/goracle package by dep, the odpi submodule inside is empty and leads to error. The reason of dropping the submodule could be found in the following link.

Update @ 2018-02-03:

The paragraph about git submodule is incorrect.

Sam Boyer wrote:

dep should be perfectly fine at pulling in git submodules in the case you describe. I just replicated what you describe here locally, and the problem isn’t submodules — it’s that there’s no Go code in github.com/go-goracle/goracle/odpi, so it can’t be imported directly.

You likely need to turn off unused-packages pruning in Gopkg.toml for that project specifically, as otherwise dep ensure will automatically remove what appears to be an unused directly (but it seems it’s actually used by cgo).

Update @ 2018-03-04:

It is found that the go-goracle/goracle package doesn’t work with dep. You could follow the issue below and check the latest update from the dep team.

Summary

  • dep is quite likely to be the official dependency management tool in the Golang community.
  • If you are starting a new Golang project, dep is good to go.
  • If you are using Glide in a legacy project. You could consider migrating to dep but i think there is no harm to keep using Glide for a while until dep is officially released.
  • In addition, missing package’s submodule may result in malfunction of your code.
  • dep is officially released.
  • dep works well on pulling git submodule.
  • Use the standard library wherever possible.
  • You can checkout this example on gitlab.com.