Wednesday, December 16, 2020

Last Post !! Moved to new site

As I am getting old 😋, it seem like I can not remember many of my earlier tech encounters. This is the place I was logging so that I refer back later in time. 😂 Among several reasons

- I had two sites: Tech blog and Personal blog  to maintain on blogger.
- It seem very difficult to post here. Rich text format. Yack.
- Searching is difficult. Now, I can search with few key stokes.

So, I have decided to do move/log my tech counters+personal items from now on to my personal site. 

mrprajesh.co.in/blog  //Simple blog with search!
mrprajesh.co.in/rajzdocs  //Quick Cheat sheet

Happy that Pandemic Year (2020) is finally coming to a end. Hope for a better year, next year 2021!

Good bye blogger!



Saturday, November 21, 2020

Climate and weather monitoring sites

Some links are outdated.

//TODO

- rainfall in all substations in TN
- Various Prediction Models

There is a private weather blogger TN weatherman
https://www.facebook.com/tamilnaduweatherman/posts

I have seen weatherman using many different tools of IMD, and other things on the internet. This list heavily influences from him and my our finding on the internet. Thanks to Google.

Doppler Radars (this has an accuracy of 30 mins)
IMD Main - http://imd.gov.in/pages/radar_main.php?adta=chn
IMD Dop - http://www.imd.gov.in/section/dwr/dynamic/dwr.htm
IMD Anim - http://www.imd.gov.in/section/dwr/img/Animation/dop_ani_main.htm



Thunder Storm Nowcast
http://nwp.imd.gov.in/ts/nowcast_new.php


Chennai Live weather (down at times)
http://59.90.28.39:8085/liveweather/index.php

Chennai Lake levels
https://chennaimetrowater.tn.gov.in/public/lake.html

Customized Rainfall information System CRIS
http://hydro.imd.gov.in/hydrometweb/(S(shcjnzfpnpbe0qedaom40lrj))/landing.aspx
Spatial>Daily

Any City Forcast past 24 hours.
http://imd.gov.in/pages/city_weather_show.php

Automatic Weather Station (AWS) and Automatic Rain Gauge(ARG) Stations
http://aws.imd.gov.in/
> ARG DATA
> Daily rainfall

PDF of Rainfall by IMD daily(5PM) and past days
http://www.imdchennai.gov.in/rsum.htm


Ground Water Level - Chennai
https://www.facebook.com/pg/CMWSSB/posts

Thursday, February 27, 2020

[Resolved] - Cannot launch Nvidia Visual Profiler on Ubuntu / Mint

My Linux Mint / Ubuntu 18.04 had openjdk 11.0.2 installed by default but it showed the below error. nvvp or Nvidia Visual Profiler (or even with nSight) requires jre-8 as a MUST.

$ nvvp
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.eclipse.osgi.storage.FrameworkExtensionInstaller (file:/usr/local/cuda-10.0/libnvvp/plugins/org.eclipse.osgi_3.10.1.v20140909-1633.jar) to method java.net.URLClassLoader.addURL(java.net.URL)
WARNING: Please consider reporting this to the maintainers of org.eclipse.osgi.storage.FrameworkExtensionInstaller
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release


Work around measures

Step 1. sudo apt install openjdk-8-jdk
Step 2. nvvp -vm /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java

That's it! tata

Monday, December 16, 2019

Building Gunrock on redhat/centOs

How to build the latest version of Gunrock (a popular GPU Graph analytic library) on Redhat/Centos based systems


Prerequisites:
- cmake 3.10+
- GCC 4.8+

Although building gunrock is easy on Ubuntu 18.04, it is not easy for other OSs.
In most of the RedHat rpms, cmake is not the latest. Sometimes we do not have root/admin rights to install. In such scenarios, below steps to build cmake/gunrock.

Build/Download cmake - 4 steps
1. Download cmake*.tar.gz from https://cmake.org/download/ 
   download either src or binary 

2. Extract

3) // do either a or b

a. if src, cd to cmake directory and do make
b. if binary, do nothing


4. To verify
./cmake-3.14.7/bin/cmake --version
displays as below
cmake version 3.14.7

CMake suite maintained and supported by Kitware (kitware.com/cmake).



Build Gunrock - 6 steps
1.) Set and export CUDA Bin path to your cuda. Mine is 9.1. Note a non-default location!

CUDA_BIN_PATH=/usr/local/cuda-9.1
export CUDA_BIN_PATH

2)
git clone --recursive https://github.com/gunrock/gunrock/
cd gunrock


3)

 In gunrock folder, edit CMakeLists.txt
 at the following lines as per your GPU card. Ours is Nvidia P100. So, I
 have changed that corresponding flag from OFF to ON. Find your card name using nvidia-smi command. 

if (CUDA_VERSION VERSION_EQUAL "8.0" OR CUDA_VERSION VERSION_GREATER "8.0")
  option(GUNROCK_GENCODE_SM60
    "ON to generate code for Compute Capability 6.0 devices (e.g. Tesla P100)"
    ON)


Note: This Step may not be needed due to this #665 pull request


4) 
mkdir build && cd build 
~/install/cmake-3.14.7/bin/cmake ..

we are using the cmake which we downloaded/build previously.
Note: Always make sure your build folder is empty before cmake. If you end up with some errors due to cmake; make sure to delete all files and folders from build before performing cmake again.


Upon success. Should display the following:
..
-- Project Added: bc
-- Project Added: bfs
-- Project Added: color
-- Project Added: geo
-- Project Added: hits
-- Project Added: knn
-- Project Added: louvain
-- Project Added: pr
-- Project Added: pr_nibble
-- Project Added: proj
-- Project Added: rw
-- Project Added: sage
-- Project Added: sm
-- Project Added: ss
-- Project Added: sssp
-- Project Added: tc
-- Project Added: vn
-- Configuring done
-- Generating done
-- Build files have been written to: /home/rajesh/tmp/gunrock/build2


5) make

Takes up to an hour or so as. If no error, then that's all. If you need only a specific algorithm (say SSSP), then read A1 below.

6) To test, cd to bin inside build dir
./sssp market ../dataset/small/test_mst.mtx

...sample output..
Distance Validity: PASS
First 40 distances of the GPU result:
[0:0 1:2 2:4 3:19 4:14 5:2147483647 6:2147483647 7:2147483647 8:2147483647 ]
First 40 distances of the reference CPU result.
[0:0 1:2 2:4 3:19 4:14 5:2147483647 6:2147483647 7:2147483647 8:2147483647 ]

[sssp] finished.
 avg. elapsed: 0.551939 ms
 iterations: 4
 min. elapsed: 0.551939 ms
 max. elapsed: 0.551939 ms
 rate: 0.016306 MiEdges/s
 src: 0
 nodes_visited: 5
 edges_visited: 9
 load time: 7.15208 ms
 preprocess time: 943.713000 ms
 postprocess time: 0.494003 ms
 total time: 945.014000 ms

YAY :-)

Appendix
A1) You could build only a specific algorithm by setting an appropriate flag during your cmake as said here in advanced compilation.
e.g. Suppose if I want SSSP alone then

/cmake-3.14.7/bin/cmake -DGUNROCK_BUILD_APPLICATIONS=OFF -DGUNROCK_APP_SSSP=ON ..
//This step builds SSSP alone when you do a make.

Errors in the past that lead to the above working steps: 

E1) CMake 3.10 or higher is required.  You are running version 2.8.12.2

E2) CMake Error at /home/rajesh/install/cmake-3.16.1-Linux-x86_64/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
  Could NOT find CUDA: Found unsuitable version "8.0", but required is at
  least "9.0" (found /usr)
Call Stack (most recent call first):
  /home/rajesh/install/cmake-3.16.1-Linux-x86_64/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:391 (_FPHSA_FAILURE_MESSAGE)
  /home/rajesh/install/cmake-3.16.1-Linux-x86_64/share/cmake-3.16/Modules/FindCUDA.cmake:1104 (find_package_handle_standard_args)
  CMakeLists.txt:41 (FIND_PACKAGE)



END- gunrock-cmd.txt

Saturday, September 14, 2019

Awesomeness of ffmpeg

ffmpeg is a powerfuly CLI tool. It has helped me many times. Assuming that it is installed.

Question: How to covert a video file to an audio file using ffmpeg in Linux

$ ffmpeg -i input-video.mp4 output-audio.mp3
$ ffmpeg -i input-video.mp4 output-audio.wav 

Note: Supports most of the audio formats.

Question: How to increase the audio quality or amplfying the audio in a video  using ffmpeg.

$ ffmpeg -i 'input-video.mp4'  -vcodec copy -af "volume=30dB" 'output-video-30db.mp4'

set volume appropriately as 10/20/30/40 DB depeding on how much audio you need to amplify in your input-video file. This works amazing well. You would not believe it.

Tata, See you.

MKV - dual or multi audio in a singe video file

MKV is one the famous video format for the movie watchers/collectors online.
WIKI says The Matroska Multimedia Container is a free, open-standard container format, a file format that can hold an unlimited number of video, audio, picture, or subtitle tracks in one file.

I am fan of the following feature. You can choose the audio language during play back. It work cools on vlc player. This would avoid creating or downloading multiple times say if we want a movie in Tamil as well as English.

I was really surprised why YouTube or vimeo did not adopt to such kind of player. That can play multiple audios/language. Maybe it will come in future some day.  Currently we have only different language subtitles for one video with single audio track. The future is single video with multiple audio tracks. This could be a boon for video creators.

It would be nice to have a player like this below.  In this below player click the headphone icon -- you can change the lang of video ENGLISH or SPANISH.

https://paellaplayer.upv.es/demos/multi-audio-tracks/

You could create a such a video. But we need a player(as above) to play such video on the web or our website. I still have not figured out how to add paella player to my site.

On a side note. How do we create a such a video. It is easy. There is a tool called MKVToolNix GUI.

Sooner or later YouTube will have this feature.

Okay. more later. Tata.

Using GNU Parallel on Linux

Sometime ago I had created a C/C++ program to find the LAN machine/IP that is powered ON or OFF in our Departmental Computing Facility[DCF]. The local IPs have changed now.

The important thing was that program pings each machines/IP address one after another. It would take a lot of time. Recently, I have seen a cmd/package called parallel  that does it in one line linux command. Cool. I can add this to my bashrc which would need only 3 key stokes to find what I wanted.

If it is not installed already, do it

$ sudo apt-get install parallel


$ parallel --halt now,success=1 ping -c1 10.21.225.{1}{2} ::: 0 1 2 ::: 1 2 3 4 5 6 7 8 9 0

This is the one line cmd. Lets break it down and see what it says.

$ parallel echo {1}{2} ::: 0 1 2 ::: 1 2 3 4 5 6 7 8 9 0

what does the above command do. Prints all the numbers from 00 to 29. i.e echo 30 times.

echo 00
echo 01
...
echo 29

parallel does the same, but the order is immaterial.

But, to my purpose I used to check $? whether  0 or 1 or something in my old script. This is done smarter here.

--halt now,success=1 means stop running jobs even if one job is succeeded. WOW!


$ parallel --halt now,success=1 ping -c1 10.21.225.{1}{2} ::: 0 1 2 ::: 1 2 3 4 5 6 7 8 9 0 > /dev/null

I use the above line which prints only the IP I wanted. YAY!
Tata. See you. 

Last Post !! Moved to new site

As I am getting old 😋, it seem like I can not remember many of my earlier tech encounters. This is the place I was logging so that I refer ...