Realizing Reflection in Golang through ORM Frame

Realizing Reflection in Golang through ORM Frame

Golang provides a mechanism to update and check the values of variants, a method to call functions of variants and internal operations they support at runtime, without knowing the exact type of these variants when compiling, which is called reflection.

basic reflection

Type and Kind

we can use function reflect.TypeOf() to get type object (reflect.Type) of any value, and then access type information through the type object.

1
2
3
4
5
6
7
8
9
10
type User struct{
Name string
Age int8
}

func main(){
user := &User{Name: "Bob", Age: 18}
typ := reflect.TypeOf(user)
fmt.Println(typ, typ.Kind())
}

running result:

1
*main.User ptr

Type refers to system’s native data type, such as int, string, bool and types defined by type keyword.

Kind refers to objects’ belonging varieties, such as int, string, bool, array, map, ptr, slice, struct.

pointer

We can get the element type pointed by the pointer through function reflect.Elem().

1
2
3
4
5
6
7
func main(){
user := &User{Name: "Bob", Age: 18}
typ := reflect.TypeOf(user)
fmt.Printf("name: %v,\nkind: %v,\n", typ.Name(), typ.Kind())
typ = typ.Elem()
fmt.Printf("element name: %v,\nelement kind: %v,\n ", typ.Name(), typ.Kind())
}

running result:

1
2
3
4
name: ,
kind: ptr,
element name: User,
element kind: struct

Value

We can use reflect.Value(data interface{}) to get the passed in parameter’s reflect.Value object.

1
2
3
pUser := &User{Name: "Bob", Age: 18}
pv := reflect.ValueOf(pUser)
fmt.Printf("typeof pv : %v\npv's type : %v\npv's value: %v\n\n", reflect.TypeOf(pv), pv.Type(), pv)

running result:

1
2
3
typeof pv : reflect.Value
pv's type : *main.User
pv's value: &{Bob 18}

reflect.Value has many same methods like reflect.Type except Name(). We can also use method Elem() to get the element type of pointer.

1
2
fmt.Printf("typeof pv.Elem() : %v\npv.Elem()'s type : %v\npv.Elem()'s value: %v\n" ,
reflect.TypeOf(pv.Elem()), pv.Elem().Type(), pv.Elem())

running result:

1
2
3
typeof pv.Elem() : reflect.Value
pv.Elem()'s type : main.User
pv.Elem()'s value: {Bob 18}

function reflect.Indirect(v Value) Value is descripted as follow: Indirect returns the value that v points to. If v is a nil pointer, Indirect returns a zero Value. If v is not a pointer, Indirect returns v.

1
2
3
4
5
6
7
pUser := &User{Name: "Bob", Age: 18}
oUser := User{Name: "Bob", Age: 18}
pv := reflect.ValueOf(pUser)
ov := reflect.ValueOf(oUser)

fmt.Println(reflect.Indirect(pv) == pv.Elem())
fmt.Println(reflect.Indirect(ov) == ov)

running result:

1
2
true
true

We can use reflect.Value’s method interface() to get the raw data.

1
2
3
oUser := User{Name: "Bob", Age: 18}
fmt.Println(reflect.TypeOf(pv.Interface()), reflect.TypeOf(pv.Elem().Interface()))
fmt.Println(pv.Elem().Interface() == oUser)

running result:

1
2
*main.User main.User
true

another example:

1
2
3
4
5
values := []interface{}{1, "str", true}
for i := 0 ; i < 3; i++{
fmt.Println(reflect.TypeOf(reflect.Indirect(reflect.ValueOf(values[i]))))
fmt.Println(reflect.TypeOf(reflect.Indirect(reflect.ValueOf(values[i])).Interface()))
}

running result:

1
2
3
4
5
6
reflect.Value
int
reflect.Value
string
reflect.Value
bool

struct

We can get detailed information of struct’ s member through function NumField() and Field() of the reflect object(reflect.Type)

Function Description
Field(i int) StructField Field returns a struct type’s i’th field. It panics if the type’s Kind is not Struct. It panics if i is not in the range [0, NumField()).
NumField() int NumField returns a struct type’s field count. It panics if the type’s Kind is not Struct.
FieldByName(name string) (StructField, bool) FieldByName returns the struct field with the given name and a boolean indicating if the field was found.
FieldByIndex(index []int) StructField FieldByIndex returns the nested field corresponding to the index sequence. It is equivalent to calling Field successively for each index i. It panics if the type’s Kind is not Struct.
FieldByNameFunc(match func(string) bool) (StructField,bool) FieldByNameFunc returns the struct field with a name that satisfies the match function and a boolean indicating if the field was found.FieldByNameFunc considers the fields in the struct itself and then the fields in any embedded structs, in breadth first order, stopping at the shallowest nesting depth containing one or more fields satisfying the match function. If multiple fields at that depth satisfy the match function, they cancel each other and FieldByNameFunc returns no match. This behavior mirrors Go’s handling of name lookup in structs containing embedded fields.

structure of StructField:

1
2
3
4
5
6
7
8
9
type StructField struct {
Name string
PkgPath string
Type Type
Tag StructTag
Offset uintptr
Index []int
Anonymous bool
}

Let’s use reflect to process sturct

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
type User struct{
Name string `json:"name"`
Age int8 `json:"age" Id:"100"`
}

func main(){
user := User{Name: "Bob", Age: 18}
typ := reflect.TypeOf(user)

// iterate the member of struct
for i := 0 ; i < typ.NumField();i++{
fieldType := typ.Field(i)
fmt.Printf("name: %v, type: %v, tag: %v\n",
fieldType.Name, fieldType, fieldType.Tag)
}
// get specific field through field name
if userAge, ok := typ.FieldByName("Age"); ok {
// get the tag
fmt.Println(userAge.Tag)
}
}

running result:

1
2
3
name: Name, type: {Name  string json:"name" 0 [0] false}, tag: json:"name"
name: Age, type: {Age int8 json:"age" Id:"100" 16 [1] false}, tag: json:"age" Id:"100"
json:"age" Id:"100"

struct tag

Tag should be written at the format of key-value pair:

1
`key1:"value1" key2:"value2"`

We can parse tag through function Get and query if the target key exists through LookUp

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
type User struct{
Name string `json:"name"`
Age int8 `json:"age" Id:"100"`
}

func main(){
user := User{Name: "Bob", Age: 18}
typ := reflect.TypeOf(user)
if userName, ok := typ.FieldByName("Name"); ok{
queryStr := []string{"json", "Id"}
for i := 0; i < 2; i++ {
if v, ok := userName.Tag.Lookup(queryStr[i]); ok{
fmt.Printf("tag %v exists, value: %v\n", queryStr[i], v)
fmt.Println("value: ", userName.Tag.Get(queryStr[i]))
}else{
fmt.Printf("tag %v does not exist\n", queryStr[i])
fmt.Println("value: ", userName.Tag.Get(queryStr[i]))
}
}
}
}

running result:

1
2
3
4
tag json exists, value: name
value: name
tag Id does not exist
value:

reflection in ORM

This part mainly refers to geektutu’s project “7 days to implement the ORM framework GeeORM from scratch with Go”. We can further realize reflection mechanism and practice through building the ORM frame.

Data Type Transformation

Data types are different between SQL and golang, and are different between database either. For the compatibility with multiple database, we should map go’s data types to database’s data types, and here comes the reflection.

take sqlite3 as a example, we use reflect.Value.Kind() to judge the type of parameters passed in and transform it into corresponding sql data type:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
func(s *sqlite3) DataTypeOf(typ reflect.Value) string{
switch typ.Kind() {
case reflect.Bool:
return "bool"
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32,
reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uintptr:
return "integer"
case reflect.Int64, reflect.Uint64:
return "bigint"
case reflect.Float32, reflect.Float64:
return "real"
case reflect.String:
return "text"
case reflect.Array, reflect.Slice:
return "blob"
case reflect.Struct:
// use reflect.Value.Interface() to get the object
// and use object.(Type) to judge the type of this object
if _, ok := typ.Interface().(time.Time); ok{
return "datetime"
}
}
panic(fmt.Sprintf("invalid sql type %s (%s)", typ.Type().Name(), typ.Type().Kind()))
}

Transformation from Go Object to Database Table

The core transformation of ORM frame: given any object, transform it into a table structure in rational database.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
// Parse is used to parse any struct object to Schema object
func Parse(dest interface{}, d dialect.Dialect) *Schema {
// if dest is a pointer, through reflect.Indirect we can get pointer's
// element type's value, and use .Type() method to reflect.Type obj
modelType := reflect.Indirect(reflect.ValueOf(dest)).Type()
schema := &Schema{
Model: dest,
Name: modelType.Name(),
fieldMap: make(map[string]*Field),
}
// use modelType.NumField() to get struct's member num
for i := 0; i < modelType.NumField(); i++ {
// use Type.Filed(i) to access struct's member variants
p := modelType.Field(i)
if !p.Anonymous && ast.IsExported(p.Name) {
field := &Field{
// get member variants' name and Type in target database
Name: p.Name,
// use reflect.Indirect() to get the object of the given pointer
// use DataTypeOf to transform the datatype into a database one
Type: d.DataTypeOf(reflect.Indirect(reflect.New(p.Type))),
}
// use Lookup to get tag's content
if v, ok := p.Tag.Lookup("geeorm"); ok {
field.Tag = v
}
// add current Field to schema
schema.Fields = append(schema.Fields, field)
schema.FieldNames = append(schema.FieldNames, p.Name)
schema.fieldMap[p.Name] = field
}
}
return schema
}

Hooks