15.02.2018       Выпуск 217 (12.02.2018 - 18.02.2018)       Статьи

Google Colab Free GPU Tutorial

Как получить бесплатный GPI Tesla K80 для Keras, Tensorflow и PyTorch.

Читать>>




Экспериментальная функция:

Ниже вы видите текст статьи по ссылке. По нему можно быстро понять ссылка достойна прочтения или нет

Просим обратить внимание, что текст по ссылке и здесь может не совпадать.

Google Colab Free GPU Tutorial

Now you can develop deep learningapplications with Google Colaboratory -on the free Tesla K80 GPU- using Keras, Tensorflowand PyTorch.

Hello! I will show you how to use Google Colab, Google’s free cloud service for AI developers. With Colab, you can develop deep learning applications on the GPU for free.

What is Google Colab?

Google Colab is a free cloud service and now it supports free GPU!

You can;

  • improve your Pythonprogramming language coding skills.
  • develop deep learning applications using popular libraries such as Keras, TensorFlow, PyTorch,and OpenCV.

The most important feature that distinguishes Colab from other free cloud services is; Colabprovides GPU and is totally free.

Detailed information about the service can be found on the faq page.

Getting Google Colab Ready to Use

Creating Folder on Google Drive

Since Colab is working on your own Google Drive, we first need to specify the folder we’ll work. I created a folder named “app” on my Google Drive. Of course, you can use a different name or choose the default Colab Notebooksfolder instead of app folder.

I created an empty “app” folder

Creating New Colab Notebook

Create a new notebook via Right click > More > Colaboratory

Right click > More > Colaboratory

Renamenotebook by means of clicking the file name.

Setting Free GPU

It is so simple to alter default hardware (CPU to GPU or vice versa); just follow Edit > Notebook settings or Runtime>Change runtime typeand select GPUas Hardware accelerator.

Running Basic Python Codes with Google Colab

Now we can start using Google Colab.

I will run some Basic Data Types codes from Python Numpy Tutorial.

It works as expected :) If you do not know Pythonwhich is the most popular programming language for AI, I would recommend this simple and clean tutorial.

Running or Importing .py Files with Google Colab

Run these codes first in order to install the necessary libraries and perform authorization.

!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
!apt-get update -qq 2>&1 > /dev/null
!apt-get -y install -qq google-drive-ocamlfuse fuse
from google.colab import auth
auth.authenticate_user()
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
import getpass
!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass()
!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}










When you run the code above, you should see a result like this:

Clickthe link, copyverification code and paste it to text box.

After completion of the authorization process,

mount your Google Drive:

!mkdir -p drive
!google-drive-ocamlfuse drive

install Keras:

!pip install -q keras

upload mnist_cnn.py file to app folder which is located on your Google Drive.

mnist_cnn.py file

run the code below to train a simple convnet on the MNIST dataset.

!python3 drive/app/mnist_cnn.py

As you can see from the results, each epoch lasts only 11 seconds.

Download Titanic Dataset (.csv File) and Display First 5 Rows

If you want to download .csv file from url to “app” folder, simply run:

!wget https://raw.githubusercontent.com/vincentarelbundock/Rdatasets/master/csv/datasets/Titanic.csv -P drive/app

You may upload your .csv files directly to “app” folder instead of wget method.

Read .csv file in “app” folder and display first 5 rows:

import pandas as pd
titanic = pd.read_csv(“drive/app/Titanic.csv”)
titanic.head(5)

Cloning Github Repo to Google Colab

It is easy to clone a Github repo with Git.

Step 1: Find the Github Repo and Get “Git” Link

Find any Github repo to use.

For instance: https://github.com/wxs/keras-mnist-tutorial

Clone or download > Copy the link!

2. Git Clone

Simply run:

!git clone https://github.com/wxs/keras-mnist-tutorial.git

3. Open the Folder in Google Drive

Folder has a same the with Github repo of course :)

4. Open The Notebook

Right Click > Open With > Colaboratory

5. Run

Now you are able to run Github repo in Google Colab.

Some Useful Tips

1. How to Install Libraries?

Keras

!pip install -q keras
import keras

PyTorch

!pip install -q http://download.pytorch.org/whl/cu75/torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl torchvision
import torch

MxNet

!apt install libnvrtc8.0
!pip install mxnet-cu80

import mxnet as mx

OpenCV

!apt-get -qq install -y libsm6 libxext6 && pip install -q -U opencv-python
import cv2

XGBoost

!pip install -q xgboost==0.4a30
import xgboost

GraphViz

!apt-get -qq install -y graphviz && pip install -q pydot
import pydot

7zip Reader

!apt-get -qq install -y libarchive-dev && pip install -q -U libarchive
import libarchive

Other Libraries

!pip install or !apt-get install to install other libraries.

2. Is GPU Working?

To see if you are currently using the GPU in Colab, you can run the following code in order to cross-check:

import tensorflow as tf
tf.test.gpu_device_name()

3. Which GPU Am I Using?

from tensorflow.python.client import device_lib
device_lib.list_local_devices()

Currently, Colab only provides Tesla K80.

4. What about RAM?

!cat /proc/meminfo

5. What about CPU?

!cat /proc/cpuinfo

6. Changing Working Directory

Normally when you run this code:

!ls

You probably see datalab and drive folders.

Therefore you must add drive/app before defining each filename.

To get rid of this problem, you can simply change the working directory. (In this tutorial I changed to app folder) with this simple code:

import os
os.chdir("drive/app")

After running code above, if you run again

!ls

You would see app folder content and don’t need to add drive/app all the time anymore.

7. “No backend with GPU available“ Error Solution

If you encounter this error:

Failed to assign a backend
No backend with GPU available. Would you like to use a runtime with no accelerator?

Try again a bit later. A lot of people are kicking the tires on GPUs right now, and this message arises when all GPUs are in use.

Reference

8. How to Clear Outputs of All Cells

Follow Tools>>Command Palette>>Clear All Outputs

9. “apt-key output should not be parsed (stdout is not a terminal)” Warning

If you encounter this warning:

Warning: apt-key output should not be parsed (stdout is not a terminal)

That means authentication has already done. You only need to mount Google Drive:

!mkdir -p drive
!google-drive-ocamlfuse drive

10. How to Use Tensorboard with Google Colab?

I recommend this repo:

https://github.com/mixuala/colab_utils

Conclusion

I think Colab will bring a new breath to Deep Learning and AI studies all over the world.

If you found this article helpful, it would mean a lot if you gave it some applause👏 and shared to help others find it! And feel free to leave a comment below.

You can find me on LinkedIn.

Last Note

This blog post will be constantly updated.

Changelog

26-01–2018

  • “insert appfoldertopath” removed
  • “downloading, reading and displaying .csv file” added
  • “Some Useful Tips” added

27–01–2018

28–01–2018

  • “Cloning Github Repo to Google Colab” added
  • “pip install mxnet” added

29–01–2018

No backend with GPU available. Error Solution added

2–02–2018

5–02–2018

  • “How to Clear Outputs of All Cells” added
  • apt-key output should not be parsed (stdout is not a terminal) warning added

11–02–2018



Лучшая Python рассылка




Разместим вашу рекламу

Пиши: mail@pythondigest.ru

Нашли опечатку?

Выделите фрагмент и отправьте нажатием Ctrl+Enter.

Система Orphus