Understanding Tensorflow using Go

原文: https://pgaleone.eu/tensorflow/go/2017/05/29/understanding-tensorflow-using-go/

 

Tensorflow is not a Machine Learning specific library, instead, is a general purpose computation library that represents computations with graphs. Its core is implemented in C++ and there are also bindings for different languages. The bindings for the Go programming language, differently from the Python ones, are a useful tool not only for using Tensorflow in Go but also for understanding how Tensorflow is implemented under the hood.

The bindings

Officially, the Tensorflow’s developers released:

• The C++ source code: the real Tensorflow core where the high & low level operations are concretely implemented.
• The Python bindings & the Python library: the bindings are automatically generated from the C++ implementation, in this way we can use Python to invoke C++ functions: that’s how, for instance, the core of numpy is implemented. The library, moreover, combines calls to the bindings in order to define the higher level API that everyone’s using Tensorflow knows well.
• The Java bindings
• The Go binding

Being a Gopher and not a Java lover, I started looking at the Go bindings in order to understand what kind of tasks they were created for.

The Go bindings

The Gopher (created by Takuya Ueda (@tenntenn). Licensed under the Creative Commons 3.0 Attributions license) 
overlapping the Tensorflow Logo.

---

The first thing to note is that the Go API, for admission of the maintainers itself, lacks the Variable support: this API is designed to use trained models and not for trainingmodels from scratch. This is clearly stated in the Installing Tensorflow for Go:

TensorFlow provides APIs for use in Go programs. These APIs are particularly well-suited to loading models created in Python and executing them within a Go application.

If we’re not interested in training ML models: hooray! If, instead, you’re interested in training models here’s an advice:

Be a real gopher, keep it simple! Use Python to define & train models; you can always load trained models and using them with Go later!

In short: the go bindings can be used to import and define constants graphs; where constant, in this context, means that there’s no training process involved and thus no trainable variables.

Let’s now start diving into Tensorflow using Go: let’s create our first application.

In the following, I suppose that the reader has its Go environment ready and the Tensorflow bindings compiled and installed as explained in the README.

Understand Tensorflow structure

Let’s repeat what Tensorflow is (kept from the Tensorflow website, the emphasis is mine):

TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them.

We can think of Tensorflow as a descriptive language, a bit like SQL, in which you describe what you want and let the underlying engine (the database) parse your query, check for syntactic and semantic errors, convert it to its private representation, optimize it and compute the results: all this to give you the correct results.

Therefore, what we really do when we use any of the available APIs is to describe a graph: the evaluation of the graph starts when we place it into a Session and explicitly decide to Run the graph within the Session.

Knowing this, let’s try to define a computational graph and evaluate it within a Session. The API documentation gives us a pretty clear list of the available methods within the packages tensorflow (shorthanded tf) & op.

As we can see, these two packages contains everything we need to define and evaluate a graph.

The former contains the functions to construct the basic “empty” structures like the Graph itself, the latter is the most important package that contains the bindings automatically generated from the C++ implementation.

However, suppose that we want to compute the matrix multiplication between AA and xx where

A=(1−12−2),x=(10100)A=(12−1−2),x=(10100)

I suppose that the reader is already familiar with the tensorflow graph definition idea and knows what placeholders are and how they work. The code below is the first attempt that a Tensorflow Python bindings user would make. Let’s call this file attempt1.go

package main

import (
	"fmt"
	tf "github.com/tensorflow/tensorflow/tensorflow/go"
	"github.com/tensorflow/tensorflow/tensorflow/go/op"
)

func main() {
	// Let's describe what we want: create the graph

	// We want to define two placeholder to fill at runtime
	// the first placeholder A will be a [2, 2] tensor of integers
	// the second placeholder x will be a [2, 1] tensor of intergers

	// Then we want to compute Y = Ax

	// Create the first node of the graph: an empty node, the root of our graph
	root := op.NewScope()

	// Define the 2 placeholders
	A := op.Placeholder(root, tf.Int64, op.PlaceholderShape(tf.MakeShape(2, 2)))
	x := op.Placeholder(root, tf.Int64, op.PlaceholderShape(tf.MakeShape(2, 1)))

	// Define the operation node that accepts A & x as inputs
	product := op.MatMul(root, A, x)

	// Every time we passed a `Scope` to an operation, we placed that
	// operation **under** that scope.
	// As you can see, we have an empty scope (created with NewScope): the empty scope
	// is the root of our graph and thus we denote it with "/".

	// Now we ask tensorflow to build the graph from our definition.
	// The concrete graph is created from the "abstract" graph we defined
	// using the combination of scope and op.

	graph, err := root.Finalize()
	if err != nil {
		// It's useless trying to handle this error in any way:
		// if we defined the graph wrongly we have to manually fix the definition.

		// It's like a SQL query: if the query is not syntactically valid
		// we have to rewrite it
		panic(err.Error())
	}

	// If here: our graph is syntatically valid.
	// We can now place it within a Session and execute it.

	var sess *tf.Session
	sess, err = tf.NewSession(graph, &tf.SessionOptions{})
	if err != nil {
		panic(err.Error())
	}

	// In order to use placeholders, we have to create the Tensors
	// containing the values to feed into the network
	var matrix, column *tf.Tensor

	// A = [ [1, 2], [-1, -2] ]
	if matrix, err = tf.NewTensor([2][2]int64{ {1, 2}, {-1, -2} }); err != nil {
		panic(err.Error())
	}
	// x = [ [10], [100] ]
	if column, err = tf.NewTensor([2][1]int64{ {10}, {100} }); err != nil {
		panic(err.Error())
	}

	var results []*tf.Tensor
	if results, err = sess.Run(map[tf.Output]*tf.Tensor{
		A: matrix,
		x: column,
	}, []tf.Output{product}, nil); err != nil {
		panic(err.Error())
	}
	for _, result := range results {
		fmt.Println(result.Value().([][]int64))
	}
}

The code is completely commented and I invite the reader to read every single comment.

Now, the Tensorflow-Python user expects that this code compiles and works fine. Let’s see if he’s right:

go run attempt1.go

Here’s what he got:

panic: failed to add operation "Placeholder": Duplicate node name in graph: 'Placeholder'

wait: what’s going on here? Apparently, there are 2 operations “Placeholder” with the same name “Placeholder”.