How to get Remote Control direction inputs commands and custom sensors readings in real time in log files

HI. We are a research group, just trying drones to fly autonomously through a deep neural network inside a tunnel. I was trying to know how can i integrate a custom sensor to pixhawk flight controller. also we will first collect the training data for our algorithm like

data%20Table

where C-1, C-2 are commands to move forward, move right etc. so if we take the first line of the data set it says that at a particular moment the D1 sensor has value 120, Azi sensor has value 152 and at that time the direction commands that was given to the uav through remote control was move left (c-3 is left for example) how can i achieve this. what is the complete work flow to create log such log files etc? Thank you