Fun Computer Vision opencv tutorials and ..


Follow

My favourite

  • Opencv tutorial people detection
  • Head people cascade download
  • Opencv tutorial optical flow
  • Opencv Video stabilization
  • Opencv car dataset download
  • Opencv tutorial Transparent mask
  • Opencv videowriter
  • Opencv FFMPEG
  • Opencv Canny edge and hough lines
  • Intel computer vision SDK the results of itseez acquisition

    The Intel defined the future usage of processing power by many acquisitions in last two years. I already wrote about the itseez and movidius. The itseez is lost somewhere in the structures of the intel but results are obviously visible. The movidius continue under intel as more separated brand with latest usb stick for accelerating the deep neural networks for embedded and mobile phone. All this activities shape intel and also architecture of the processors, size of processors register, floating point  registers optimization, vector architectures and much smarter parallelism than ever before. 
    intel computer vision SDK

     The Intel® CV SDK

    Basically is package of the product related to video processing, computer vision, machine learning and lots of optimization inside these packages done by Intel. The optimization also comes with better deployment for systems that combined many hardware like CPU, GPU and FPGA. The every devices has advantage in some stages of image processing pipeline. 

    Opencv, OpenVX integration

    Intel SDK try to merge OpenVX and Opencv more closely, which is great.. OpenVX is cross platform acceleration for computer vision application that doesn't necessary required the high power CPU and GPU. The Opencv itself coming with cuda support and OpenVX integrations opens the access to hw like movidius accelerators for deep neural network, small IOT devices, Cameras and more.  Huge application potential from home security to smart cars and devices. Opencv is more community driven project and OpenVX os more hardware vendors API integration focusing on low level optimization. The tight and simple merging inside Intel Computer Vision SDK makes lots of senses from Intels HW, acquisitions and future perspective.

    Cuda sample flow VS Opencv OpenVX integration

    The following two examples try to compare the working program from CUDA perspectives and also The Intel CV SDK. For example CUDA devices transfers data in from host CPU to devices GPU by following flow. The flow more than running cuda code sample!! 
    
    //Allocating the input output memory
    int *in = (int*)malloc(size);
    int *out=(int*)malloc(size);
    cudaMalloc((void **)&inAddressGPU,InputSize);
    cudaMalloc((void **)&outAddressGPU,OutputSize);
    //Copy from HOST (in) to DEVICES (inAdressGPU), where parallel magic will happened 
    //pass results back to HOST from outAddressGPU to out by memcpy, memory copy
    cudaMemcpy(inAddressGPU,in,size,cudaMemcpyHostToDevice)
    //Do somethink lunch kernel on GPU atd atd
    //Copy results back
    cudaMemcpy(out,outAddressGPI,size,cudaMemcpyDevicesToHost)
    free(in)
    free(out)
    cudaFree(inAddressGPU)
    cudaFree(outAddressGPU)
    


    In comparison flow between the Opencv and OpenVX

    Mat RGBimage;
    vx_imagepath_addressing_t mappingTemplate;
    // mapping the sizes
    mappingTemplate.dim_x
    mappingTemplate.dim_y
    mappingTemplate.stride_x
    mappingTemplate.stride_y
    
    //pointer to data
    
    void *ptr[] = {RGBimage.data};
    
    //Create the VX_imagevx_image image=vxCreateImageFromHandle(ctx,…,& mappingTemplate,ptr, VX_MEMORY_TYPE_HOST);
    cv::cvtColor(RGBimageR, RGBimage, cv::COLOR_BGR2RGB);
    vxFunctionOver(image, ............);
    
    
    // results from from Opencv and OpenVX over one image, data in memory by different tools. 
    In some specific mapping we can do the Opencv operation and OpenVX operation over the same image. Vx_image point to same data in mamery as RGBimage MAT which is valid when the VX_MEMORY_TYPE_HOST memory is on host sideIn GPU cuda is always necessary to dopy from host to device and back in example above. This was just philosophy and architecture behind the code. The small pieces that Cuda and technology like OpenVX are something hard to deal with.  

    Intel Computer Vision DSK and Convolution neural network

    As many others framework Intel extensions for convolution neural networks comes with integration with model trained by different Software like TensorFlow or Coffe. The training is done on state of the art software maintained by large science community over the world and the feed forward part over the network without learning is deployed into many devices by tools like this Convolutional Neural Network IP kernels OpenVX extensions. 

    The additional software to work with Opencv, OpenVX, Convolutional Nural Networks OpenVX extensions is Intel's visual Algorithm Designer. Where the functionality of computer vision pipeline could be assembled in visual GUI and check the results of each edges as well as advanced profiler for the performance tuning of each stage of computer vision pipeline. 

    Another pieces is Intel's Deep Learning Framework, API for training models and Intel's standard piece of software which is Intel's Media SDK, which is great tool for codecs development and measurement of performance in encoding and decoding the video by different mechanism. 

    More about the tool here





    IP camera in cloud, web and mobile comparison 2017 for home security and small business 

    Comparison of various ip cameras ready for cloud and mobile. I would mainly focus on evaluation of smart features presented by different ip camera producers. Price for smart features and storage recording from different company is big part of evaluation. In comparison against the fashioned brands, I would also talk about the traditional cameras and the adaption in the 2017 market.  Lets start. No watching the red roofs of beautiful Prague. Now we can start.
    cloud video storage

    NEST camera

    This cameras are ready 24/7 monitor in 1080 HD what ever you want. There are two main version. One outdoor with function listen and talk like in most electronic bells and one in indoor version. Nest is able to record up to 30 days in cloud, but also price is according to that plan higher. The camera is able to capture important moments and recap the day in just selected important events. The outdoor camera have 130 degree wide angle which is great from my perspective as well as 8x zoom to catch the details.  

    Pricing for cloud and additional service is 10$/month for basic 10 day history, intelligent alerts and activity zones. The 30 day video history recording is charged by 30$ per month. From my perspective all the cameras of this types are useless without this subscription. You need to count with initial camera purchase and additional cloud storage service. Honestly, i do not know if there is some free storage, probably yes but be careful and check this.  Amazon Cloud Cam have for example 24 hours cloud storage for free and alerts. 

    Amazon Cloud Cam

    This is almost the same as in case on Nest. The 24/7 recording with smart events and recording. There is also included Two-way audio with build in microphones and speaker, intelligent alerts, person detection and night vision. The 24 hour storage is included free. There are no advanced functions, just notifications without people detection. The 7 day storage with all the Person detection, zones and alerting cost something like  7$ per month, 14 days 10$ and 20$ for 30 days cloud storage. It is obvious, the video in HD is large to save as well as transferred video through internet. 


    Foscam

    Smart cameras by Foscam like C1 C2 and many others looks pretty much the same as cameras above.  There is much more cameras for home security and indoor purpose. The Foscam comes with cloud management and control of their cameras, where the video is available live and also recorded, depends on Cloud plan you pay.  The camera comes with motion detection, which basically separate background from foreground. The methods are usually very sensitive for light conditions but the cameras are senses by Passive infrared sensors to keep eyes on the objects and prevent the false alerts by changing light conditions. Be careful, not all the cameras have this sensors but C1 and C2 and mainly indoor cameras. Foscam cloud storage plans are 5$ per 7 day back recording and 12.99$ for 30 day of recording. 


    Axis

    More traditional company with great cameras. There is various indoor and outdoor cameras of all kind with various view points. Most of the cameras needs little bit complicated settings but not two much. Axis offers hosted services for mobile access and web management in flexible custom solution for small business. The axis have lots of applications build in inside the cameras. Motion detection, regions defenders and many others are build in basic firmware inside the camera web interface. There is also available various applications for your cameras like People counter, 3D people counter, Queue monitors, motions guard, cross line detection. All the apps could be connected to AXIS store report tool, which is WEB BI, dashboard that collecting the data from security cameras and create reports and dashboards mainly for retails and advertisement marketers. 

    Homeboy

    This home security wire-free camera records audio, video with night lightning features. No wire means that the camera running on battery for 2-3 months cycles in typical usage. WiFi tech sends 5-30 second video clips, not streaming. This is what confused me. Looks like there is no live stream access to the camera. Motions alerts that trigger messages and alarm are also included. This cameras looks more like snap stories recorder into, where clips are provided in mobile apps. Why not. Nobody have a time to check all day video back in time. But i want to be sure that all accidents are safely recorded for insurance purpose. Check more about this, if you have a plan to buy one. 


    Other solutions cloud for cloud recording cameras

    This will be part of the next post. Cloud storage and apps for different types of cameras. Smart boxes that connect your cameras into the cloud without to much effort. 

    Opencv in Docker, Build app in docker tutorial

    How to run and build application based on Opencv in Docker container is purpose of this tutorial. The process described bellow is how to install Docker engine in windows machine, by command line download base Ubuntu image, install Opencv inside the container and commit changes to create own image to save your installation to add what ever opencv app you want to scale in Docker.. Cool ? No magic. Really
    docker opencv app

    Install Docker in windows

    The important think is that windows running docker machine in Hyper-V virtual machine available under Windows 10. If you select this installation you can have troubles with Virtual Box no longer work after Hyper-V is set to enable.. There is option use also installation based on Oracle Virtual Box but i select easy install using the first options which is nicely integrated with Windows 10 and brings me advantage of docker development under ubuntu on windows machines.. 

    For the installation just follow the step by step tutorial on first link for Hyper-V based installation


    Download and install Links !Read text before!
    To download Docker for Windows Hyper-V use this link
    To download Docker for Windows using Oracle VirtualBox link


    Play with docker

    After the easy installation you should have available icon in Windows like in following image. 

    Docker windows opencv
    This allows you to control your docker engine like start stop and update, all the work is done by easy commands in CMD command line.. Do not worry of command line.. It is easy like to write email.. 
    Docker opencv

    The installed docker is now available through cmd line.. 
    Docker help tutorial
    You can simply verify by issue command as me docker help and if you have succesfully installed docker you should see the same output of possible command options. 

    Docker basics 

    Just briefly about docker.. I try to use simplest word to describe this. By issue command docker ps you can see that i do not have anything set up. The main components are docker container and image. 

    Docker

    Docker image

    It is easy, Image is something like CD, all you need to install is there ready to be used by specific application. The most often the application you want to develop is included in this image. For example, basic ubuntu installation. This image could be extend to satisfy specific needs of developer. 

    Docker contained

    The container is running image instance. I have one image with ubuntu and http server inside. I could run from one image several containers with exactly same settings. This allows me easy scalable  deployment of my application from 10 users  on one container to 10 milion in hundred container of same app and setting from this base image. CD that could be installed several times is just container..


    Install Opencv in ubuntu docker image

    You can use different kind of images to start. I would like to recommend ubuntu, where we install our opencv library. The process is that you take some base image from already exist repository. This is just set up that you run container and image is basically downloaded in background. After you install opencv and prerequisites and commit changes inside container. This commit create your unique image with your saved changes (installed) opencv and save locally.. If you skipp this step your changes inside container are lost..  

    C:\docker run -it ubuntu bash

    By issue this command you run ubuntu basic container and start ubuntu bash command line.. You can exit bash command line inside ubuntu containe by type EXIT.
    I just issue different command docker run -it ID_of_image bash because i have already image installed. But after you issue C:\docker run -it ubuntu bash you will be inside linux like on picture bellow. 
    opencv docker

    Now install opencv follow the documentation.

    All is done in bash ubuntu command line. Again no mystery and who is afraid of command line is just because he used to work in different environment. But it is necessary and behind is nothing really hard.  Now we follow exactly same installation for Linux distribution, I got somethink for Debian here just a short version but you will be find with follow approach.. 

    1. install Required dependencies 
    We are root that mean sudo is not necessary, just copy this 3 lines one by one and follow the several instructions. Basically sometimes is write Y to command line. Nothink else. 

    apt-get install build-essential

    apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev

    apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev



    Now you have everything you need to build opencv. Now issue this command to download source code from git. This will create you the opencv folder, and after by cd jump into opencv folder. By issue mkdir build you create new directory. Name as build! Jump into by cd build.. You can start install right now.

    git clone https://github.com/opencv/opencv.git

    cd opencv 

    mkdir build

    cd build

    Now you are in build folder. This one is empty but is under the opecv where the all source code are located. Here just run following command for basic configuration.. Now take some time and after you will see some overview what is available to be installed. For example avlibs to use video captures and other opencv vide options.. Do not forgot the dots on next line :)

    cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local ..

    Now is time to issue another command which build your opencv from source based on cmake configuration. By type simple make in linux bash.

    make                      OR    make -j4 to run in parallel 4 build jobs.

    But just make is quite fine. Now this steps take several minutes. 10 to 40 depends how much power and -j jobs you run to make opencv. The last is to install opencv into the system by make install. 

    make install

    Done, your library are ready and you can develop your app under Linux (But this will be in next tutorial) in docker (/usr/local/lib) check the screen shot below for location cd /usr/local/lib , and list them by ls to display this.
    docker ubuntu opencv

     NOW we need to save what is in our container. When we are saving new container we create new image from we are starting containers.  

    !!Do not issue exit yet.. save first!! 

    Save changes mean - commit and create new image

    This create new image with all the changes and allow you start container from this image with all the configuration from previous step. Maybe run 1000 containers to serve the video application for millions of customers.  Open new commad line windows like on image. Run command that display running container docker -ps.

    Docker image

    Now from new cmd line save the container by this command in c:\> docker commit your_container_id name/description:version ,BE sure that c3f279d17e0a is the same container id  obtain by docker -ps command.. Like in example. 

    docker commit c3f279d17e0a ubuntu/image:version3

    This take some time and create your local image. Than could be again started by same command.
    C:\docker run -it ubuntu/image:version3 bash 

    Next steps for docker/opencv tutorials 

    Build the app inside the environment which is just prepare and use visual studio with remote build in ubuntu container.. The next tutorial already in progress.

    Reference for docker

    The best easy manual for docker is here







    As a big fan of OPENCV 3.3.0 There is what is new!

    Some my notes about new released. Based on changelog and released notes. 

    Deep neural network module is now accelerated with improved performance also moved into the main repository branch under opencv/modules/dnnThis module is also no more available in contrib branch. There was improved loading models from Troch and TensorFlow and many performance improvements. 

    Support of
    • Caffe 1
    • TensorFlow
    • Torch/PyTorch
    Interesting is available layer types list 
    • AbsVal
    • AveragePooling
    • BatchNormalization
    • Concatenation
    • Convolution (including dilated convolution)
    • Crop
    • Deconvolution, a.k.a. transposed convolution or full convolution
    • DetectionOutput (SSD-specific layer)
    • Dropout
    • Eltwise (+, *, max)
    • Flatten
    • FullyConnected
    • LRN
    • LSTM
    • MaxPooling
    • MaxUnpooling
    • MVN
    • NormalizeBBox (SSD-specific layer)
    • Padding
    • Permute
    • Power
    • PReLU (including ChannelPReLU with channel-specific slopes)
    • PriorBox (SSD-specific layer)
    • ReLU
    • RNN
    • Scale
    • Shift
    • Sigmoid
    • Slice
    • Softmax
    • Split
    • TanH

    And even more

    Advanced core performance AVX, AVX2, SSE and NEON
    New Python and C++ samples DNN C++ and Python api


    Another mainly performance improvements is 15% speed according to IPPICV from 2015.12 to 2017.2 version upgrade

    Opencv C++ 11 ready 

    I am interesting about news C++ 11 support. This should speed up development a bit. Tested on fedora distribution and Opencv should be build with -DENABLE_CXX11=ON. I will try on windows. I do not expect any problem. Build Opencv 3.3 with Visual Studio 2017 and CXX11 support.
    Examples like

    parallel_for_
    And dummy auto containers :)
    auto A = Mat_<double>({0, -1, 0, -1, 5, -1, 0, -1, 0}).reshape(1, 3);

    Opencv hardware-accelerated video encoding/decoding

    This is big in my eyes. Continue doing this.. Encoding and decoding of raw H.264 and MPEG1/2 video streams is supported, media containers are not supported yet.

    Big thanks to all contributors of OPENCV 3.3.0 to doing this great lib even better. 

     

     


    Machine learning by Andrew Ng

    machine learning

    I would like to summarize couple of thoughts about this famous coursera course. I just guess what you want to know. There is couple of facts important for me.

    Facts about

    • Link to course here
    • Price 75 dollars, not sure 
    • 11 weeks, 25 assignment from that 8 programming exercises in Matlab Octave.
    • You need to pass all assignment to get Certificate
    • Not a problem to finish in 5 weeks.
    • Video lectures, PDF, discussion, Matlab Octave background materials, data sets, and much more, teachers to help you any time
    • As a student, you focus on critical elements to really understand the machine learning. Not how to use and build model in TensorFlow like in some other course.. 
    • In Matlab, Octave you design and programming critical parts to be able understand how  machine learning works. Not python call of some black box. Real Math behind..
    Again and again constructing different Cost function for different type of problems to be able to evaluate how good your model is. 
    Gradient descent in different application and debugging if you are on right track. Advance method of optimization and different types of learning batch, stochastic gradient descent and many other for large scale machine learning.  

     
    • Some course about deep learning focus for example to using tensorflow and design model in tensorFlow. Not to essentials of ML implementations and    
    What is included

    • Linear regression to predict or maybe better to understand evaluation of some metrics based on some observation. 
    • Logistic regression to be able assigned something to class or not 0, 1
    • Neural networks, Cost function, Learning by back propagation and handling of over fitting.
    • SVM one of the best machine learning result of all time. 
    • K mean to be able observe several cluster center in unsupervised learning  
    • Evaluation, error according to recognize the problem before your learning ends nowhere. This practical part to recognize over fitting and high bias is everywhere.  It is right. The concept is quite known. The experience in ML is hard to learn almost harder than the theory itself. 
    • Unsupervised learning, K mean and PCA 
    • Quite lots of application example, Building and design of recommendation system and anomaly detection

    Basically like any others ML bachelor course in little bit extended focus in neural networks application and implementation details. Which is plus and very good starting point. 


    What is missing ? There is no topics like. This topics you should study after you fully understand the previous topics quite well, anyway.

    • Recurrent Neural networks 
    • Deep learning 
    • Convolution neural network 
    Thanks for this course.. I got my certificate on Linkedin. My knowledge are far behind but still this is great extension and recapitulation of already known thinks towards better understanding.. 

    Go and take it.. 

    ad