Go

Go Go Go

尴尬了,这篇的日期写的是2018-01-03,实际上是近三年前的帖子了。Go 1.9发布的时候就信誓旦旦的要进行学习。结果到今天还是无疾而终的样子,半途而废了好久。捞上来,看看这次可以坚持多久。

Mixin 大群部署完全教程 拿这个练习,前后端一起端。

静态站点

Serving Static Sites with Go

package main

import (
  "log"
  "net/http"
  "os"
)

func main() {
  // fs := http.FileServer(HTMLDir{http.Dir("./static")})
  fs := http.FileServer(http.Dir("./static")) //absolute path if necessary
  http.Handle("/", fs)

  log.Println("Listening on :3000...")
  err := http.ListenAndServe(":3000", nil)
  if err != nil {
    log.Fatal(err)
  }
}

type HTMLDir struct {
	d http.Dir
}

// https://stackoverflow.com/a/57281956/1087122
func (d HTMLDir) Open(name string) (http.File, error) {
	// Try name as supplied
	f, err := d.d.Open(name)
	if os.IsNotExist(err) {
		// Not found, try with .html
		if f, err := d.d.Open(name + ".html"); err == nil {
			return f, nil
		}
	}
	return f, err
}
  • Hugo生成静态站点
  • 使用上面的go脚本(保存为blog.go,编译go build blog.go,生成blog可执行文件)即可做为web服务器
  • sudo certbot --nginx自动更新配置(确保:nginx中server_name有对应的真实有效域名,并且域名配置了对应的公网IP(域名解析)),还自动添加了301跳转
  • 欢迎访问备份blog站点不想注册
  • 设置为service服务,下面的内容保存为/etc/systemd/system/blog.service,执行systemctl restart blog.service期待服务
  • 执行journalctl -u blog.service查看此服务的log信息
[Unit]
Description=Blog Daemon
After=network.target

[Service]
Type=simple
ExecStart=/home/geb/blog
Restart=on-failure

[Install]
WantedBy=multi-user.target

server {

    server_name blog.buxiangzhuce.com;
    index index.html index.htm;
    charset utf-8;

    location / {
        proxy_pass http://127.0.0.1:1313;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Port $server_port;
    }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/blog.buxiangzhuce.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/blog.buxiangzhuce.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = blog.buxiangzhuce.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

   server_name blog.buxiangzhuce.com;
    listen 80;
    return 404; # managed by Certbot

}

Wire

go wire and wire blog

前提:已经将$GOPATH/bin目录添加到环境变量$PATH 安装go get github.com/google/wire/cmd/wire

wire tutorial

  • Goland 提示 wire.go doesn't match to garget system. File will be ignored by build tool

在设置–> Go –> Build Tags & Vendoring中的Custom tags里指定要用的编译tag,例如 wireinject。参考指定编译tag编译go build -tags编译官方手册

  • 执行wire命令时提示go list stderr <<go: finding module for package pattern=. >>
go list stderr <<go: finding module for package pattern=. >>
wire: packages not found
wire: generate failed

官方issue #125中的解决方案——

# 先升级 go/packages,可以添加 -v参数看到升级了哪些packages
go get -u golang.org/x/tools/go/packages
# 再重新安装 wire
go get github.com/google/wire/cmd/wire

解释是这样的——

So Wire uses golang.org/x/tools/go/packages behind the scenes in order to gather all of the files to parse. go/packages is specifically designed to be agnostic to source code layout (e.g. modules versus GOPATH workspaces), so it shells out to the go tool to obtain the information that it needs. Since this requires cooperation from both your installed go tool and the go/packages version, an outdated go/packages library can mess up the communication and report bad results to Wire (which is what happened here).

In the Go modules future, your built version of Wire would have used a known-tested version of go/packages and this issue would not have occurred.

执行完之后,可以正常生成wire_gen.go文件。可以先执行wire check检查是否符合编译条件。

执行完一次wire命令之后,再次需要更新wire_gen.go文件时,执行go generate命令即可

undefined: InitializeEvent, 使用wire生成注入代码之后,编译时需要带上对应的代码。

执行 go build默认使用了上次的配置? go build main.go wire_gen.go可生成可执行文件

wire guide

依赖两个相同的provider?

go ldflags

Using ldflags with go build

ld stands for linker, the program that links together the different pieces of the compiled source code into the final binary.

ldflags, stands for linker flags. It passes a flag to the underlying Go toolchain linker, cmd/link, that allows you to change the values of imported packages at build time from the command line.

使用方式为:

# 格式
go build -ldflags="-flag"
# 示例
go build -ldflags="-X 'package_path.variable_name=new_value'"
# 实例
go build -ldflags="-X 'main.Version=v1.0.0'"
#复杂实例
go build -v -ldflags="-X 'main.Version=v1.0.0' -X 'app/build.User=$(id -u -n)' -X 'app/build.Time=$(date)'"
  • 外层使用双引号,确保传递的flag中的内容即使包含空格也不截断命令;
  • key-value值使用单引号
  • 要改变的变量需要是包级别的string类型变量。不能是const类型
  • 变量是否export都可以(大小写开头的变量都支持)

进一步可以使用nm工具查找编译文件中的symbols。(包名中不能包含非ASCII码,引号"和百分号%

在make文件中使用——

main.go

package main

var (
	version string
	date    string
)

func init() {
	if version == "" {
		version = "no version"
	}
	if date == "" {
		date = "(Mon YYYY)"
	}
}

func main() {
	println(version, date)
}

makefile:

version=0.0.1
date=$(shell date -j "+(%b %Y)")
exec=a.out

.PHONY: all

all:
	@echo " make <cmd>"
	@echo ""
	@echo "commands:"
	@echo " build          - runs go build"
	@echo " build_version  - runs go build with ldflags version=${version} & date=${date}"
	@echo ""

build: clean
	@go build -v -o ${exec}

build_version: check_version
	@go build -v -ldflags '-X "main.version=${version}" -X "main.date=${date}"' -o ${exec}_${version}

clean:
	@rm -f ${exec}

check_version:
	@if [ -a "${exec}_${version}" ]; then \
		echo "${exec}_${version} already exists"; \
		exit 1; \
	fi;

Setting Go variables from the outside

go run命令中也可以直接使用(因为会默认先执行go build)

go run -ldflags="-X main.who CloudFlare" hello.go

fmt string

Go by Example: String Formatting

package main

import (
	"fmt"
	"os"
)

type point struct {
	x, y int
}

func main() {

	p := point{1, 2}

	// v for verbs? value?
	// {1 2}
	fmt.Printf("%v\n", p)

	// include the struct’s field names.
	// {x:1 y:2}
	fmt.Printf("%+v\n", p)

	//  prints a Go syntax representation of the value
	//  main.point{x:1, y:2}
	fmt.Printf("%#v\n", p)

	// print the type of a value
	// main.point
	fmt.Printf("%T\n", p)

	// Formatting booleans
	// false
	fmt.Printf("%t\n", false)

	//  base-10 formatting.
	// 123
	fmt.Printf("%d\n", 123)

	// binary formatting
	// 1110
	fmt.Printf("%b\n", 14)

	// prints the character corresponding to the given integer.
	// !
	fmt.Printf("%c\n", 33)

	// provides hex encoding.
	// 1c8
	fmt.Printf("%x\n", 456)

	//  formatting options for floats. For basic decimal formatting use %f.
	// 78.900000
	fmt.Printf("%f\n", 78.9)

	// format the float in (slightly different versions of) scientific notation
	// 1.234000e+08
	fmt.Printf("%e\n", 123400000.0)
	// 1.234000E+08
	fmt.Printf("%E\n", 123400000.0)

	// print basic string
	// "string"
	fmt.Printf("%s\n", "\"string\"")

	// double-quote strings
	// "\"string\""
	fmt.Printf("%q\n", "\"string\"")

	// renders the string in base-16, with two output characters per byte of input
	// 6865782074686973
	fmt.Printf("%x\n", "hex this")

	// print a representation of a pointer
	// 0xc00000a210 (address)
	fmt.Printf("%p\n", &p)

	// specify the width of an integer, use a number after the % in the verb. By default the result will be right-justified and padded with spaces.
	// |    12|   345|
	fmt.Printf("|%6d|%6d|\n", 12, 345)

	// restrict the decimal precision at the same time with the width.precision syntax
	// |  1.20|  3.45|
	fmt.Printf("|%6.2f|%6.2f|\n", 1.2, 3.45)

	// left-justify, use the - flag
	// |1.20  |3.45  |
	fmt.Printf("|%-6.2f|%-6.2f|\n", 1.2, 3.45)

	// format string
	// |   foo|     b|
	fmt.Printf("|%6s|%6s|\n", "foo", "b")

	// |foo   |b     |
	fmt.Printf("|%-6s|%-6s|\n", "foo", "b")

	// return string
	s := fmt.Sprintf("a %s", "string")
	fmt.Println(s)

	// format+print to io.Writers other than os.Stdout using Fprintf.
	fmt.Fprintf(os.Stderr, "an %s\n", "error")

}

闭包函数传参:

Passing parameters to function closureUsing goroutines on loop iterator variables

for i := 0; i < 3; i++ {
    go func() {
        fmt.Println(i)
    }()
}
//result: 3 3 3

for i := 0; i < 3; i++ {
    go func(v int) {
        fmt.Println(v)
    }(i)
}
// result: 0, 1, 2

How to write Go code

重新更新了环境,使用go1.14.4开始,即使在window环境上,安装也只需要解压、更新环境变量(如果使用cgo的话,还需要gcc环境,机器上已经有了,但目前以我这水平应该还用不到)就可以立即上手了。

How to write Go code实践——

  • 创建工作目录,后续在工作目录进行操作
  • 使用go mod init example.com/user/hello进行初始化,生成go.mod文件,包含了包名和使用的版本名
  • 使用go install进行安装,三种等价操作。go install ., go install example.com/user/hello
  • 在本地定义引用的包信息:目录名作为引入的内容,其中的方法使用大写字母开头自动exported,使用go build就可以将引用包安装到本地。之后就可以在应用中使用
  • 引入remote的包会自动去下载远程的包。通过go installgo buildgo run都可以触发下载动作,并会更新go.mod文件
  • 使用 go test进行测试

注意事项:

  1. 可以使用go env -w GOBIN=/path/to/bingo env -u GOBIN进行变量声明和修改
  2. 在cmd环境下可以临时设置代理,持续到窗口关闭,前提是本地已经有了proxyset http_proxy=http://127.0.0.1:7890, set https_proxy=http://127.0.0.1:7890
  3. 使用go test ./...可以自动遍历当前工程所有文件夹下的test文件

要用到go module,参考这两篇——Go Modules 终极入门干货满满的 Go Modules 和 goproxy.cn

Gocker Docker

用 Go 从头实现一个迷你 Docker — Gocker
原文: Containers the hard way: Gocker: A mini Docker written in Go
源码:Gocker

这个看起来有点帅啊,而且代码不是很多的样子,结合耗子叔的这几篇一起理解更有帮助——

DOCKER基础技术:LINUX NAMESPACE(上)
DOCKER基础技术:LINUX NAMESPACE(下)
DOCKER基础技术:LINUX CGROUP
DOCKER基础技术:AUFS
DOCKER基础技术:DEVICEMAPPER

结构体对应数据绑定

Go Echo: get started, Go Echo: basic features跟着这个例子写一个简单的登录demo。

趁热打铁,第三篇新鲜出炉Go Echo: custom Binder

  • param tag 对应路径参数;
  • query tag 对应 URL 参数;
  • json tag 对应 application/json 方式参数;
  • form tag 对应 POST 表单数据;
  • xml tag 对应 application/xml 或 text/xml;
// 表述结构体对应数据绑定时,对应的tag和字段。 如,json类型的name字段;form类型的name字段
type User struct {
 Name string `query:"name" form:"name" json:"name" xml:"name"`
 Sex  string `query:"sex" form:"sex" json:"sex" xml:"sex"`
}

顺带安装一下很好用的http调试工具httpie,python写得,依赖python3.6以上版本(本地的环境已经很混乱得啥都有, 使用py --version可以调用python3版本),直接使用 pip install --upgrade httpie安装成功了。

没有进行数据绑定时,传递xml类型数据时,被认为是Content-Type: text/plain;;指定了绑定规则后可以识别并转换为Content-Type: application/json; charset=UTF-8

正则限制

https://groups.google.com/forum/#!topic/golang-nuts/7qgSDWPIh_E

that (?!re) regular expression is not supported neither in re2 nor in Google Go. Is there any chance that its support will be implemented in future releases? (it is supported at least in Ruby, Python, Java, Perl)

https://www.reddit.com/r/programmingcirclejerk/comments/5ml6yj/golangs_standard_regex_library_doesnt_have/

go-issues:18868 regexp: support lookaheads and lookbehinds

Golang doesn’t support positive lookbehind assertion?

支持的场景: https://github.com/google/re2/wiki/Syntax要求makes a single scan over the input and runs in O(n) time

The lack of generalized assertions, like the lack of backreferences, is not a statement on our part about regular expression style. It is a consequence of not knowing how to implement them efficiently. If you can implement them while preserving the guarantees made by the current package regexp, namely that it makes a single scan over the input and runs in O(n) time, then I would be happy to review and approve that CL. However, I have pondered how to do this for five years, off and on, and gotten nowhere.

包含在一传字符串中的手机号,类似abcd13566778888xyz这样的手机号码前后不包含数字的正则在Go语言里是不支持的。因为无法一次扫描并在O(n)的时间里完成。

(?=re) before text matching re (NOT SUPPORTED) (?!re) before text not matching re (NOT SUPPORTED) (?<=re) after text matching re (NOT SUPPORTED) (?<!re) after text not matching re (NOT SUPPORTED)

go vendor 依赖

低版本下对于vendor目录的查找逻辑:

  • 需要将项目建立在$GOPATH目录下的src目录(前提)
  • 查找当前工程下的vendor目录(vendor tree) (如果项目没有在$GOPATH目录下,这一步将会被忽略)go issue #14566
  • $GOROOT目录下查找(from $GOROOT)
  • $GOPATH目录下查找(from $GOPATH)

archive

Go release 1.9 version on 24 August 2017. I’m the 31,365 people to start it on github and just get started to learn it :).

Go 1.9 is released. Here are all the blogs about go.

Go: Ten years and climbing Go十年 BIG: Blockchain In Go

VS环境设置:

Go tools that the Go extension depends on

Ctrl + b: 显示/隐藏侧边栏 Ctrl + j: 显示/隐藏Panel控制栏

update project

# Add Hugo and its package dependencies to your go src directory.
go get -v github.com/gohugoio/hugo

$GOPATH 目录约定有三个子目录:

  • src 存放源代码(比如:.go .c .h .s等)
  • pkg 编译后生成的文件(比如:.a)
  • bin 编译后生成的可执行文件(为了方便,可以把此目录加入到 $PATH 变量中,如果有多个gopath,那么使用${GOPATH//://bin:}/bin添加所有的bin目录)
$ go env
set GOARCH=amd64
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=F:\Tools\gopath\
set GORACE=
set GOROOT=F:\Tools\go
set GOTOOLDIR=F:\Tools\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=G:\TempFolder\userTemp\go-build565356171=/tmp/go-build -gno-record-gcc-switches
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config

go get golang.org/x 包失败解决方法

方案由于限制问题,国内使用 go get 安装 golang 官方包可能会失败

go get -v golang.org/x/tour/pic
Fetching https://golang.org/x/tour/pic?go-get=1
https fetch failed: Get https://golang.org/x/tour/pic?go-get=1: dial tcp 216.239.37.1:443: i/o timeout
package golang.org/x/tour/pic: unrecognized import path "golang.org/x/tour/pic" (https fetch: Get https://golang.org/x/tour/pic?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)

golang 在 github 上建立了一个**镜像库**,如 https://github.com/golang/net 即是 https://golang.org/x/net 的镜像库

获取 golang.org/x/net 包,其实只需要以下步骤:

mkdir -p $GOPATH/src/golang.org/x
cd $GOPATH/src/golang.org/x
git clone https://github.com/golang/net.git

## example
➜  src mkdir -p golang.org/x
➜  src cd golang.org/x
➜  x git clone https://github.com/golang/tour.git
Cloning into 'tour'...
remote: Counting objects: 2092, done.
remote: Total 2092 (delta 0), reused 0 (delta 0), pack-reused 2092
Receiving objects: 100% (2092/2092), 895.19 KiB | 133.00 KiB/s, done.
Resolving deltas: 100% (1329/1329), done.
## now you can import "golang.org/x/tour/pic" 

其它 golang.org/x 下的包获取皆可使用该方法

或者使用软连接的方式代替

git clone https://github.com/golang/net.git $GOPATH/src/github.com/golang/net

git clone https://github.com/golang/sys.git $GOPATH/src/github.com/golang/sys

git clone https://github.com/golang/tools.git $GOPATH/src/github.com/golang/tools

ln -s $GOPATH/src/github.com/golang $GOPATH/src/golang.org/x1

go test

同事的项目本地执行没有问题,线上跑go test的时候一直无法通过,build 失败。

最终定位原因:

  • 环境需要安装gcc环境
  • 打开golang的环境变量 CGO_ENABLED="1"

环境默认是打开了CGO的,但执行go test时会报gcc错误。为了不安装gcc环境,强制修改了这个变量(还尝试了半天修改的方法)

这是因为甚至是go test ./...时,有些报会使用到 “C混合编译”,需要注意这俩个关键因素。

go import

Understanding Dependency Management in Go Understanding Vendoring:

In order to be able to fully understand how vendoring works we must understand the algorithm used by Go to resolve import paths, which is the following:

  1. Look for the import at the local vendor directory (if any)
  2. If we can’t find this package in the local vendor directory we go up to the parent folder and try to find it in the vendor directory there (if any)
  3. We repeat step 2 until we reach $GOPATH/src
  4. We look for the imported package at $GOROOT
  5. If we can’t find this package at $GOROOT we look for it in our $GOPATH/src folder

go mod

Go mod 使用
告别GOPATH,快速使用 go mod(Golang包管理工具)

go.mod文件中定义了当前项目对应的module名称,如golang.gebitang.com/my/module

对于pkg/util/log的包,当前项目中使用import时,可以使用下面的方式进行引入。go mod模块会自动进行转换

import (
    "golang.gebitang.com/my/module/pkg/util/log"
)

go mod将依赖统一放到GOPATH下的pkg下的pkg下面,并且支持不同版本(使用@vMajor.minor.path)的格式管理

usage of go mod vendor

The go mod vendor command constructs a directory named vendor in the main module’s root directory that contains copies of all packages needed to support builds and tests of packages in the main module.

As you append to a slice, its capacity doubles in size every time it exceeds its current capacity.

 
comments powered by Disqus