anko
Well-known member
Haha. Like that.gwatpe said:
-
If you have not please join our official My Outlander PHEV Facebook Group!
Mitsubishi Outlander PHEV Facebook Group
Extracting useful data from your OBDII port
- Thread starter anko
- Start date
Help Support Mitsubishi Outlander PHEV Forum:
gwatpe
Well-known member
Torque PRO has a PID for L/100km petrol consumption. Works OK for when the PHEV is moving, but returns a zero for when it is stationary. Will be revising the PID to use L/hr and speed and calculate consumption. Petrol consumption with ICE operating when stopped at lights is not accumulated. Will be tweaking more PID logged to make it easier to review the logged data and even calc say kWh in and out of the battery to really see how much work the battery is doing.
anko
Well-known member
Torque Pro does have a PID for this. But it is a calculated value, not a value retrieved directly from the ECU of the car. This is what the maker of TP says about it. Might be useful for you to take notice of:
Maker of Torque Pro said:
gwatpe
Well-known member
I have been using the enhanced mpg calculations and this has brought the L/100km more in line with actual fuel used as well as the calc in the car, but the 0 reading with the ICE powered ON with the car stationary will still cause errors in OBD data for PHEV operation in certain modes. External calcs still need to have petrol consumption, even when the car is not moving.
zzcoopej
Well-known member
zzcoopej said:
Getting close, from the latest Beta....
anko
Well-known member
As of now,we are also able to log B-level settings:
So, it should become easier to correlate max regen levels to different B-levels
I have been chasing Charge, Save and Normal mode, but so far I only found the push of the buttons in the raw OBDII data, not the actual state.
So, it should become easier to correlate max regen levels to different B-levels
I have been chasing Charge, Save and Normal mode, but so far I only found the push of the buttons in the raw OBDII data, not the actual state.
zzcoopej
Well-known member
anko said:
This will be very interesting to find out, thanks again for your perseverance in seeking out all the PHEV secrets.
gwatpe
Well-known member
The logging of OBD data with Torque PRO produces many MB/hr of data for just a handful of logged PID.
I have found that logging of data, faster than once a second seems a bit pointless, as I see many duplicate data points, so clearly the logging is faster than the PID are updated. I am presently logging 28 PID. There are GPS and ACCElerometer data that take up many columns. The only way to get data time stamps is to log as a single file.
My sequence of PID will be quite different to what another driver may want to log, so any data crunching will need to be adaptable to any sequence of PID in the file.
The file is a .csv format and may will say EXCEL or ACCESS will work with that. Trouble is the data file contains a fair bit of corrupted data, with stops and restarts, and excess column headers that seem to get recorded. I do use Excel to fix the simple problems. The main problem is what do you do when the data appears a fuzz of noise like this.
and the section of interest was ten minutes around 16:45.
This took under 1 minute, with no database.
from raw data that looked like this
The white trace is elevation, and the PHEV throttle response under CC @ 100kph, the petrol consumption and the battery power, and the generator power interactions seem a jumble, until filtered.
I had been critical with filtering data, but by using integration instead of a rolling average filter, the data is not time biased.
I am still developing my windows app and will add more functions.
I also have click inversion of data, that works with the filtering and hope to add some $/km type calculations, that will require adding petrol and electrical data, with the effects of the battery and terrain, so it will be some work yet.
I have found that logging of data, faster than once a second seems a bit pointless, as I see many duplicate data points, so clearly the logging is faster than the PID are updated. I am presently logging 28 PID. There are GPS and ACCElerometer data that take up many columns. The only way to get data time stamps is to log as a single file.
My sequence of PID will be quite different to what another driver may want to log, so any data crunching will need to be adaptable to any sequence of PID in the file.
The file is a .csv format and may will say EXCEL or ACCESS will work with that. Trouble is the data file contains a fair bit of corrupted data, with stops and restarts, and excess column headers that seem to get recorded. I do use Excel to fix the simple problems. The main problem is what do you do when the data appears a fuzz of noise like this.
and the section of interest was ten minutes around 16:45.
This took under 1 minute, with no database.
from raw data that looked like this
The white trace is elevation, and the PHEV throttle response under CC @ 100kph, the petrol consumption and the battery power, and the generator power interactions seem a jumble, until filtered.
I had been critical with filtering data, but by using integration instead of a rolling average filter, the data is not time biased.
I am still developing my windows app and will add more functions.
I also have click inversion of data, that works with the filtering and hope to add some $/km type calculations, that will require adding petrol and electrical data, with the effects of the battery and terrain, so it will be some work yet.
anko
Well-known member
Can see seed of each individual wheel now Useful? Dunno. Fun? Yep.
Useful? Dunno. Fun? Yep.gwatpe
Well-known member
Extracting Useful data from the industrial noise type environment of the PHEV is important in the process of interpretation of how PHEV systems interact with each other for a particular driving need.
Anko has made it possible for a commercial OBD app, like Torque PRO, that has logging facilities, to be able to work with the particular MMC computers in the PHEV, using an android device. His efforts cannot be understated, and he now has a method, of obtaining and recording data that has been hidden behind walls of secrets, that would normally only be available to a service dealer with custom MMC tools like a MUT3. Collaboration with others, including the developer of EvBatMon, with beta testing across the planet will hopefully result in apps that give any PHEV owner, access to OBD data, with a relatively cheap adapter and an app on an android smart device.
The standard PID list in Torque PRO, does allow for some data, but as I have found, even this data requires some processing to be useful.
The OBD data for the petrol tank level is near useless as live data and becomes more noisy as the tank empties. The level of filtering to make the OBD data useful is between seconds and minutes, depending on the parameter. This has an impact on presentation of OBD numbers live on a display screen, and anyone interested in using a live app to aid with their driving style will benefit from an app that incorporates the right level of filtering of the data prior to display.
The post processing of logged data allows more filtering options. I use averaging of averaging of data about every data point, in multiple passes vs time to provide as much filtering as an individual PID seems to need.
I don't want to spend too much time on a spreadsheet, to crunch some numbers. I prefer a click to select some data for a particular date and then just click to select the data I am interested in viewing, and then clicking to display a graph ON or OFF, and clicking to get a clearer picture, and then zooming in for closer inspection, with the ability to just click again, to reload some new data
I spent another life using a spreadsheet to crunch numbers, for reports and presentations. I am now time poor and pondering over mountains of data, selecting useful bits has to be effortless and fast. More time for driving. :lol:
Anko has made it possible for a commercial OBD app, like Torque PRO, that has logging facilities, to be able to work with the particular MMC computers in the PHEV, using an android device. His efforts cannot be understated, and he now has a method, of obtaining and recording data that has been hidden behind walls of secrets, that would normally only be available to a service dealer with custom MMC tools like a MUT3. Collaboration with others, including the developer of EvBatMon, with beta testing across the planet will hopefully result in apps that give any PHEV owner, access to OBD data, with a relatively cheap adapter and an app on an android smart device.
The standard PID list in Torque PRO, does allow for some data, but as I have found, even this data requires some processing to be useful.
The OBD data for the petrol tank level is near useless as live data and becomes more noisy as the tank empties. The level of filtering to make the OBD data useful is between seconds and minutes, depending on the parameter. This has an impact on presentation of OBD numbers live on a display screen, and anyone interested in using a live app to aid with their driving style will benefit from an app that incorporates the right level of filtering of the data prior to display.
The post processing of logged data allows more filtering options. I use averaging of averaging of data about every data point, in multiple passes vs time to provide as much filtering as an individual PID seems to need.
I don't want to spend too much time on a spreadsheet, to crunch some numbers. I prefer a click to select some data for a particular date and then just click to select the data I am interested in viewing, and then clicking to display a graph ON or OFF, and clicking to get a clearer picture, and then zooming in for closer inspection, with the ability to just click again, to reload some new data
I spent another life using a spreadsheet to crunch numbers, for reports and presentations. I am now time poor and pondering over mountains of data, selecting useful bits has to be effortless and fast. More time for driving. :lol:
anko
Well-known member
Thanks for the kind words. But lets be careful with filtering out too much (what you call) noise before presenting data. For fuel level type data, of course, you want to average out values read. But for other types of data, you may not want to smoothen things out.gwatpe said:
Remember the graphs I presented, that showed how the PHEV plays with the Charge Current to control speed at low SOCs and plays with the throttle to control speed at high SOCs? Too much smoothening-out would have hidden that effect from us.
I am not saying we should not smooth out data here and there. I am just saying we should be careful when doing so
gwatpe
Well-known member
Trip costs are a high priority for most drivers, and knowing how many kWh and litres were used for the morning school run compared to the afternoon, and the summer cost vs the winter cost for the same run are questions a PHEV driver might like to have answered. With some logging of the OBD data, we may have a record of what the car did, but the useful cost is a bit harder to get. The PHEV does not have a PID for that. Any tool for analyzing the data has to be easy to use and fast, with minimal user effort, or else the tool will simply not be used routinely.
My windows program can give a breakdown of a trip with litres and kWh and distance and a cost for the trip, as $/100km, for comparisons morning to afternoon, or season to season for the same trip, by clicking on a calendar, and selecting the time portion of interest to display.
It seems that many drivers have the same driving pattern day after day, and are getting conflicting returned economies, or range. If the PID for temperature is also considered with economy and battery data, then some understandings of car operations will possibly become more clear.
here is an example of a trip.
This return trip had an average running cost of $5 per 100km.
Will only need to confirm the calcs vs actual measured data from the petrol pump and kWh meter.
My windows program can give a breakdown of a trip with litres and kWh and distance and a cost for the trip, as $/100km, for comparisons morning to afternoon, or season to season for the same trip, by clicking on a calendar, and selecting the time portion of interest to display.
It seems that many drivers have the same driving pattern day after day, and are getting conflicting returned economies, or range. If the PID for temperature is also considered with economy and battery data, then some understandings of car operations will possibly become more clear.
here is an example of a trip.
This return trip had an average running cost of $5 per 100km.
Will only need to confirm the calcs vs actual measured data from the petrol pump and kWh meter.
Tried EVBatmon on my Android tab with obd2 connector - EV SERVICE REQUIRED! engine management light on all the time.
Connector on it's way back to Amazon, last time I fiddle with anything.
I want my MK1 Cortina back.
Chris
Connector on it's way back to Amazon, last time I fiddle with anything.
I want my MK1 Cortina back.
Chris
anko
Well-known member
Please, if you are not really interested in CANBUS and OBD tools, do yourself a favour and skip this post. If you decide to continue, do me a favour and don't ask me "why would you". As the only answer I can give you is "not because I have to, but because I can" :mrgreen:
Until a little while ago, I have been capturing OBDII data by sending requests and capturing responses to the requests. There are two disadvantages to this approach:
- It is relatively slow, as a single request may take up to 250 msecs to be fulfilled and for a decent dashboard you may easily need 10 different requests. The result is a sweep time of 2 - 3 seconds. Not very attractive.
- You are transmitting requests ONTO the CANBUS. Compared to the fast amount of data found on the CANBUS at any moment, it is very, very little and should be of no consequence. But yet, there is a tiny risk of interfering.
So, I have been investigating an alternative approach (for example also used by the CANION app for the iMiEV): listening in on the ongoing communication between different ECUs and sensors in the car. The idea is to write a program that monitors all traffic and buffers the last version of each type of data item, so requests from Torque Pro (or any other OBD tool) can be quickly fulfilled by this program using the buffered data. My first tests with very elaborate dashboards show sweep times of (if I have to guess) a quarter of a second. Very promising.
During one of the first road tests I did, I captured about 6.5M data items (filling a 622 MB CSV file) during a 90 minute trip. The amount of data was rather intimidating, but nevertheless I took a crack at it. I was able to recognise quite some interesting parameters. But the parameters I was able to identify seemed mostly related to the 'conventional' part of the car. I could not find anything that seemed related to for example the rear E-motor or the generator. Then I remembered the CANBUS diagrams that have been posted here earlier. Closer study of these diagrams revealed that the main CANBUS is effectively split into two with the PHEV ECU acting as a firewall between the two halves. This is done to limit the amount of traffic on each of the halves. Only data that is needed on both sides is allowed to pass the PHEV ECU from one half to the other. As a consequence, from the OBD diagnostic port you can only listen in on the traffic on one half of the main CANBUS. The half that harbours the more conventional ECUs.
So, this weekend, I used the Russian workshop manuals to locate and identify the wires for the other CANBUS half. I managed to tap into them at the connector that connects the rear E-motor ECU to the CANBUS. To this tap, I connected a female OBD connector, effectively creating a second OBD diagnosis port. Today and yesterday, I have been able to monitor traffic on the second half of the CANBUS with some interesting successes.
One of these successes is that I am now able to monitor SOC at 1/100th (!) of a percent. I can just watch it go up and down while driving .... I can witness SOC go down during coasting, because power is used for eliminating e-drag and such. Very nice.
Below graphs give another idea of the kind of information I was able to retrieve by monitoring the existing communication on the second half CANBUS. I have selected some graphs that focus on the generator. These graphs reflect the last few kms or 7.5 minutes of my 40 km daily commute into the office this morning. I almost made it pure EV this morning, but I fell a few 100 meters short. So, the engine started up when I could already see the office building. Th engine ran for about 65 seconds, 50 of which were the warm up phase, where the engine was not really contributing, and 15 were actually useful. After the 15 seconds I arrived at the last traffic lights and the remainder of the trip was pure EV in the parking lot at very low speed.
Requested Torque for Front and Rear Motor and Generator (approx. 42.000 frames during 7.5 minutes)
This graph shows the PHEV ECU asking the front (red trace) and rear (blue trace) e-motors to propel or slow down the car by requesting positive or negative torque. In the middle section, you see the PHEV ECU asking the generator to start the engine (positive spike in the green trace). Then you see that, during a warm-up phase of 50 seconds, the engine is nearly idling, as the generator is asked to produce only very little resistance (negative torque). In the last 15 seconds, the generator is asked to produce some more resistance / generate a bit more electric power.
Generator Actual Torque and RPM (approx. 42.000 data frames captured during 7.5 minutes)
The purple trace is Actual Torque produced by the generator, positive when starting the engine or negative when generating electric power. Actual Torque follows Requested Torque very closely. The red trace is generator RPM, roughly three times engine RPM, I guess. The brown trace appears to be a 'running state' or such. What the green trace is, I have no idea about yet. I am especially curious about that one, as it is showing activity even before the generator is woken up by the PHEV ECU.
Generator Torque and RPM (zoomed in)
Same as above but zoomed in into the more interesting part of the 7.5 minutes.
Generator Currents (approx. 21.000 data frames captured during 7.5 minutes)
This diagram shows Current 1 and Current 2 on the AC motor / generator (red and blue traces). also, it shows (if I understand correctly) the difference in percentage between target current and real current, for both cCurrent 1 and Current 2 (the green and blue traces hugging the 100 mark all the time). The other two traces I suspect to be coil temperatures, but I have not been able to map them exactly.
Generator Currents (zoomed in)
Same as above but zoomed in into the more interesting part of the 7.5 minutes.
Some undefined stuff (approx. 4.400 data frames captured during 7.5 minutes)
If you have any idea what this could be, by all means, let me know
Until a little while ago, I have been capturing OBDII data by sending requests and capturing responses to the requests. There are two disadvantages to this approach:
- It is relatively slow, as a single request may take up to 250 msecs to be fulfilled and for a decent dashboard you may easily need 10 different requests. The result is a sweep time of 2 - 3 seconds. Not very attractive.
- You are transmitting requests ONTO the CANBUS. Compared to the fast amount of data found on the CANBUS at any moment, it is very, very little and should be of no consequence. But yet, there is a tiny risk of interfering.
So, I have been investigating an alternative approach (for example also used by the CANION app for the iMiEV): listening in on the ongoing communication between different ECUs and sensors in the car. The idea is to write a program that monitors all traffic and buffers the last version of each type of data item, so requests from Torque Pro (or any other OBD tool) can be quickly fulfilled by this program using the buffered data. My first tests with very elaborate dashboards show sweep times of (if I have to guess) a quarter of a second. Very promising.
During one of the first road tests I did, I captured about 6.5M data items (filling a 622 MB CSV file) during a 90 minute trip. The amount of data was rather intimidating, but nevertheless I took a crack at it. I was able to recognise quite some interesting parameters. But the parameters I was able to identify seemed mostly related to the 'conventional' part of the car. I could not find anything that seemed related to for example the rear E-motor or the generator. Then I remembered the CANBUS diagrams that have been posted here earlier. Closer study of these diagrams revealed that the main CANBUS is effectively split into two with the PHEV ECU acting as a firewall between the two halves. This is done to limit the amount of traffic on each of the halves. Only data that is needed on both sides is allowed to pass the PHEV ECU from one half to the other. As a consequence, from the OBD diagnostic port you can only listen in on the traffic on one half of the main CANBUS. The half that harbours the more conventional ECUs.
So, this weekend, I used the Russian workshop manuals to locate and identify the wires for the other CANBUS half. I managed to tap into them at the connector that connects the rear E-motor ECU to the CANBUS. To this tap, I connected a female OBD connector, effectively creating a second OBD diagnosis port. Today and yesterday, I have been able to monitor traffic on the second half of the CANBUS with some interesting successes.
One of these successes is that I am now able to monitor SOC at 1/100th (!) of a percent. I can just watch it go up and down while driving .... I can witness SOC go down during coasting, because power is used for eliminating e-drag and such. Very nice.
Below graphs give another idea of the kind of information I was able to retrieve by monitoring the existing communication on the second half CANBUS. I have selected some graphs that focus on the generator. These graphs reflect the last few kms or 7.5 minutes of my 40 km daily commute into the office this morning. I almost made it pure EV this morning, but I fell a few 100 meters short. So, the engine started up when I could already see the office building. Th engine ran for about 65 seconds, 50 of which were the warm up phase, where the engine was not really contributing, and 15 were actually useful. After the 15 seconds I arrived at the last traffic lights and the remainder of the trip was pure EV in the parking lot at very low speed.
Requested Torque for Front and Rear Motor and Generator (approx. 42.000 frames during 7.5 minutes)
This graph shows the PHEV ECU asking the front (red trace) and rear (blue trace) e-motors to propel or slow down the car by requesting positive or negative torque. In the middle section, you see the PHEV ECU asking the generator to start the engine (positive spike in the green trace). Then you see that, during a warm-up phase of 50 seconds, the engine is nearly idling, as the generator is asked to produce only very little resistance (negative torque). In the last 15 seconds, the generator is asked to produce some more resistance / generate a bit more electric power.
Generator Actual Torque and RPM (approx. 42.000 data frames captured during 7.5 minutes)
The purple trace is Actual Torque produced by the generator, positive when starting the engine or negative when generating electric power. Actual Torque follows Requested Torque very closely. The red trace is generator RPM, roughly three times engine RPM, I guess. The brown trace appears to be a 'running state' or such. What the green trace is, I have no idea about yet. I am especially curious about that one, as it is showing activity even before the generator is woken up by the PHEV ECU.
Generator Torque and RPM (zoomed in)
Same as above but zoomed in into the more interesting part of the 7.5 minutes.
Generator Currents (approx. 21.000 data frames captured during 7.5 minutes)
This diagram shows Current 1 and Current 2 on the AC motor / generator (red and blue traces). also, it shows (if I understand correctly) the difference in percentage between target current and real current, for both cCurrent 1 and Current 2 (the green and blue traces hugging the 100 mark all the time). The other two traces I suspect to be coil temperatures, but I have not been able to map them exactly.
Generator Currents (zoomed in)
Same as above but zoomed in into the more interesting part of the 7.5 minutes.
Some undefined stuff (approx. 4.400 data frames captured during 7.5 minutes)
If you have any idea what this could be, by all means, let me know
anko
Well-known member
Also allows me to do this ....
https://www.youtube.com/watch?v=Gfci2XKyrwM
https://www.youtube.com/watch?v=Gfci2XKyrwM
GreyBigFoot
Well-known member
Awesome topic here
So Anko, more developments there?
I'm looking into building my own monitoring solution, which would be the conventional request/response method.
I've looked at the first posts and there is a lot valuable information there (thanks anko).
Still, would it be possible to have a complete list of the requests/responses formats available for the PHEV?
So Anko, more developments there?
I'm looking into building my own monitoring solution, which would be the conventional request/response method.
I've looked at the first posts and there is a lot valuable information there (thanks anko).
Still, would it be possible to have a complete list of the requests/responses formats available for the PHEV?
anko
Well-known member
Haven't done much with this lately, other than trying to discover same data via bus monitoring, rather than question / response. And even that has been about a year ago.
GreyBigFoot
Well-known member
Ah... that's too bad :/
I'm doing some progress here as you can see bellow:
Have read all your posts, including this one:
http://www.myoutlanderphev.com/forum/viewtopic.php?f=10&t=1655
Still I'm trying to gather a list of requests/responses for all that precious information we are all looking for.
Not trying to do some money with this, whatever that will come out from this it will be available for the comunity.
Doing this just for the fun and because I'm a freak for data and analysis (i'm a data analyst by profession).
Stay tuned!
I'm doing some progress here as you can see bellow:
Have read all your posts, including this one:
http://www.myoutlanderphev.com/forum/viewtopic.php?f=10&t=1655
Still I'm trying to gather a list of requests/responses for all that precious information we are all looking for.
Not trying to do some money with this, whatever that will come out from this it will be available for the comunity.
Doing this just for the fun and because I'm a freak for data and analysis (i'm a data analyst by profession).
Stay tuned!
GreyBigFoot
Well-known member
Well, thanks to anko's valuable work I've already managed to roll some numbers.
GPS, bluetooth and ODB interface are all working well and I'm able to request and process nicely the BMU ECU frame.
Still much work to do in the layouts/UI and trip logging, but everything is coming nicely
Stay tuned, I will be probably releasing an early functional version of this that will display all the essential battery data.
GPS, bluetooth and ODB interface are all working well and I'm able to request and process nicely the BMU ECU frame.
Still much work to do in the layouts/UI and trip logging, but everything is coming nicely
Stay tuned, I will be probably releasing an early functional version of this that will display all the essential battery data.
Great work "GreyBigFoot"
For me it will be nice to be able to have logging of data .. something that EvBatMon is a bit weak.
If you also are able to send some error code .. like the failure that happen when the fuse of the fuel pomp is disabled .. it will be a great work around for have pure EV mode on any PHEV
For me it will be nice to be able to have logging of data .. something that EvBatMon is a bit weak.
If you also are able to send some error code .. like the failure that happen when the fuse of the fuel pomp is disabled .. it will be a great work around for have pure EV mode on any PHEV
