Streaming data between apps
#1
Senior Member
Thread Starter
Join Date: Nov 2003
Location: Rockport, TX
Posts: 647
Likes: 0
Received 0 Likes
on
0 Posts
Car: 1980 Jeep CJ7
Engine: AMC 360 with GM TBI
Transmission: TF727 set for kill...let's get it on!!!
Streaming data between apps
Just a thought and I am sure someone has tried this before. A friend and I were wondering how difficult it would be to stream data from WinALDL or TunerPro RT into MS Excel in real time. We are playing around with the idea of real-time tuning of a C3 ECM using an automated process that would stream data from WinALDL or TunerPro into a pre-formatted spreadsheet. This would likely work best while emulating so that changes could be seen as the engine is running.
Any suggestions?
Any suggestions?
#2
Junior Member
Join Date: Oct 2004
Location: Austin, TX
Posts: 17
Likes: 0
Received 0 Likes
on
0 Posts
I guess I'm the "other half" of this consortium.
I'm not only thinking of streaming data between apps, I'm looking for something more ambitious in the long run. I'd like to write an application that can read the ALDL directly, run some (convergent) tuning equations based on what it reads and talk directly to a PROM emulator in real time. No other applications in the way.
I do have several years of experience writing real time embedded software doing big math so that part doesn't scare me. I just need a couple of things: a) advice on the best place to find the comm specs and protocols for ALDL readers and the common PROM simulators; b) time and motivation to crank out the code; c) someone more familiar than I am with the engine control equations and how they interact based on the sensor data (or a good book that talks about them); d) someone who knows the ALDL readers and PROM simulators inside and out so I can bounce ideas off them.
The idea would be to have a windows based application that had a few control buttons and some real time status indidators. One would only have to buy the ALDL cable, the PROM simulator, plug them in and tweak the controls to tune while driving down the road (or sitting on a dyno). Once a good tune is found, an image can then be burnt to a PROM and the prom put in the ECM.
That's the idea anyway. And it's not much more developed at this point. I have the ECM maps so I can make sense out of them fairly easily. I just haven't started cranking out any designs yet. (It doesn't help that my engine is still on a stand and not yet in my Jeep so I don't have anything to work with in the garage yet.)
I'm not only thinking of streaming data between apps, I'm looking for something more ambitious in the long run. I'd like to write an application that can read the ALDL directly, run some (convergent) tuning equations based on what it reads and talk directly to a PROM emulator in real time. No other applications in the way.
I do have several years of experience writing real time embedded software doing big math so that part doesn't scare me. I just need a couple of things: a) advice on the best place to find the comm specs and protocols for ALDL readers and the common PROM simulators; b) time and motivation to crank out the code; c) someone more familiar than I am with the engine control equations and how they interact based on the sensor data (or a good book that talks about them); d) someone who knows the ALDL readers and PROM simulators inside and out so I can bounce ideas off them.
The idea would be to have a windows based application that had a few control buttons and some real time status indidators. One would only have to buy the ALDL cable, the PROM simulator, plug them in and tweak the controls to tune while driving down the road (or sitting on a dyno). Once a good tune is found, an image can then be burnt to a PROM and the prom put in the ECM.
That's the idea anyway. And it's not much more developed at this point. I have the ECM maps so I can make sense out of them fairly easily. I just haven't started cranking out any designs yet. (It doesn't help that my engine is still on a stand and not yet in my Jeep so I don't have anything to work with in the garage yet.)
#3
Junior Member
Join Date: Oct 2004
Location: Austin, TX
Posts: 17
Likes: 0
Received 0 Likes
on
0 Posts
Ok, I'm digging through most of the stickied posts with resource links. It's going to take me a while to digest it all. If anyone wants to chime in with advice, espeically with advanced topics that might be beyond shade tree mechanic levels, by all means do so.
#4
TGO Supporter
Join Date: Jul 2000
Posts: 812
Likes: 0
Received 1 Like
on
1 Post
Engine: 5.7L V8
Transmission: 700R4
I think the source to Craig's software is available, and there is also Freescan if you are looking for ALDL streams. I'd be interested in helping with this, but I'm busy with some hacs, and <A HREF="http://wasabi.dynu.com:8080/wiki/index.php/EcmEdit">EcmEdit</A>. I want to get those projects out of the way before I start any others.
Alex
Alex
#5
Junior Member
Join Date: Oct 2004
Location: Austin, TX
Posts: 17
Likes: 0
Received 0 Likes
on
0 Posts
Originally posted by AlexJH
I think the source to Craig's software is available, and there is also Freescan if you are looking for ALDL streams. I'd be interested in helping with this, but I'm busy with some hacs, and <A HREF="http://wasabi.dynu.com:8080/wiki/index.php/EcmEdit">EcmEdit</A>. I want to get those projects out of the way before I start any others.
Alex
I think the source to Craig's software is available, and there is also Freescan if you are looking for ALDL streams. I'd be interested in helping with this, but I'm busy with some hacs, and <A HREF="http://wasabi.dynu.com:8080/wiki/index.php/EcmEdit">EcmEdit</A>. I want to get those projects out of the way before I start any others.
Alex
While I'm a good software engineer, I'm still a novice when it comes to engine control. I'm looking at 3-4 months minimum of digesting information and learning about the complexities of engine calibration. Besides, I need to get my engine into the Jeep and get it back on the road so I have my own working test bed and that's going to take 6 more months.
The rough plan is to develop a nice C++ program on windoze and Linux to do this on-the-fly calabration of the proms. I want to make it modular so it will work with a variety of proms. I figure there has to be a way to monitor all the sensors to recalibrate the tables in the PROM on the fly given a baseline map for a starting point. I'm betting it's easier to do that big math on fast laptops. The 8 bit controllers on some of these ECMs aren't really fast enough to do that level of calibration. Besides, they don't really have a way to update their own PROM tables without serious hacks.
So if you have ideas and want to pitch in, I'd welcome the help. I may try starting an email list (or even a small forum site) to share data, plans and discussions for whoever wants to pitch in.
But like I say, I'm just starting to get my feet wet. It's going to take a few months to ramp up to the point where we come up with the basic outline for the program design.
Last edited by derf; 01-06-2005 at 11:01 AM.
#6
TGO Supporter
Join Date: Jan 2000
Location: In your ear. No, the other one.
Posts: 1,861
Likes: 0
Received 0 Likes
on
0 Posts
Car: '89 Trans Am WS6
Engine: 350 TPI
Transmission: T5WC
Axle/Gears: 3.08 posi
Originally posted by derf
a) advice on the best place to find the comm specs and protocols for ALDL readers and the common PROM simulators;
a) advice on the best place to find the comm specs and protocols for ALDL readers and the common PROM simulators;
Just about every GM OBDI ALDL spec is there. Have at it.
#7
TGO Supporter
Join Date: Jan 2000
Location: In your ear. No, the other one.
Posts: 1,861
Likes: 0
Received 0 Likes
on
0 Posts
Car: '89 Trans Am WS6
Engine: 350 TPI
Transmission: T5WC
Axle/Gears: 3.08 posi
Originally posted by derf
The 8 bit controllers on some of these ECMs aren't really fast enough to do that level of calibration. Besides, they don't really have a way to update their own PROM tables without serious hacks.
The 8 bit controllers on some of these ECMs aren't really fast enough to do that level of calibration. Besides, they don't really have a way to update their own PROM tables without serious hacks.
Trending Topics
#8
TGO Supporter
Join Date: Jul 2000
Posts: 812
Likes: 0
Received 1 Like
on
1 Post
Engine: 5.7L V8
Transmission: 700R4
Originally posted by derf
The rough plan is to develop a nice C++ program on windoze and Linux to do this on-the-fly calabration of the proms.
The rough plan is to develop a nice C++ program on windoze and Linux to do this on-the-fly calabration of the proms.
I'm a linux weenie.
Originally posted by derf
So if you have ideas and want to pitch in, I'd welcome the help. I may try starting an email list (or even a small forum site) to share data, plans and discussions for whoever wants to pitch in.
So if you have ideas and want to pitch in, I'd welcome the help. I may try starting an email list (or even a small forum site) to share data, plans and discussions for whoever wants to pitch in.
#9
Junior Member
Join Date: Oct 2004
Location: Austin, TX
Posts: 17
Likes: 0
Received 0 Likes
on
0 Posts
Originally posted by Mangus
I'd think this would be nearly impossible (you aren't going to get an ECM to burn a chip in-socket - or at least not in a way that everyone can use easily). I think the best way to go about this is to do the real time tuning by way of an emulator (Romulator, AutoProm, Ostrich, Prominator Pro).
I'd think this would be nearly impossible (you aren't going to get an ECM to burn a chip in-socket - or at least not in a way that everyone can use easily). I think the best way to go about this is to do the real time tuning by way of an emulator (Romulator, AutoProm, Ostrich, Prominator Pro).
Oh, and thanks for the link. That will really help.
#10
TGO Supporter
Join Date: Jan 2000
Location: In your ear. No, the other one.
Posts: 1,861
Likes: 0
Received 0 Likes
on
0 Posts
Car: '89 Trans Am WS6
Engine: 350 TPI
Transmission: T5WC
Axle/Gears: 3.08 posi
Yes, you'd want to drive the emulation hardware in your own app (rather than through mine). See the Romulator web site. They list the protocol for communication. The Prominator is a little more difficult to interface with, being parallel and all (Bill!).
#11
Junior Member
Join Date: Oct 2004
Location: Austin, TX
Posts: 17
Likes: 0
Received 0 Likes
on
0 Posts
Originally posted by AlexJH
I'm a linux weenie.
I'm a linux weenie.
I've been using Linux since kernel 1.2.7 off and on. I know a lot of DIY tuners are also Linux geeks so the program has to be available for them. However, a Windoze version would appeal to a broad segment of the population too. I've done cross platform development (NT and Linux) for device drivers before so this shouldn't be too much of a challenge, especially if we design it to be modular. It's not too hard to swap out the comm modules and the calls to create and run the window interface if it's designed to do that from the begining.
Originally posted by AlexJH
Feel free to use my Wiki site: http://wasabi.dynu.com:8080/wiki/index.php/Main_Page
Feel free to use my Wiki site: http://wasabi.dynu.com:8080/wiki/index.php/Main_Page
#12
Junior Member
Join Date: Oct 2004
Location: Austin, TX
Posts: 17
Likes: 0
Received 0 Likes
on
0 Posts
Originally posted by Mangus
Yes, you'd want to drive the emulation hardware in your own app (rather than through mine). See the Romulator web site. They list the protocol for communication. The Prominator is a little more difficult to interface with, being parallel and all (Bill!).
Yes, you'd want to drive the emulation hardware in your own app (rather than through mine). See the Romulator web site. They list the protocol for communication. The Prominator is a little more difficult to interface with, being parallel and all (Bill!).
Man, I really am going to have to start some kind of web site/blog/forum to get all this organized.
I think I'm going to start with a manual system where all the streaming data shows up and you can manually tweak everything in the same app. I'll use that to get the interfaces debugged and working. Then I'll start adding the "self tuning" portions. That seems to be a good rough heading to start out on...
Last edited by derf; 01-06-2005 at 12:53 PM.
#13
TGO Supporter
Join Date: Jan 2000
Location: In your ear. No, the other one.
Posts: 1,861
Likes: 0
Received 0 Likes
on
0 Posts
Car: '89 Trans Am WS6
Engine: 350 TPI
Transmission: T5WC
Axle/Gears: 3.08 posi
Yeah, you'll probably want to do that. What you propose is certainly not simple, nor will it be quick (regardless of your programming talent). Go for it. DIY-PROM'ers love innovation! :-D
#14
Junior Member
Join Date: Oct 2004
Location: Austin, TX
Posts: 17
Likes: 0
Received 0 Likes
on
0 Posts
Originally posted by Mangus
Yeah, you'll probably want to do that. What you propose is certainly not simple, nor will it be quick (regardless of your programming talent). Go for it. DIY-PROM'ers love innovation! :-D
Yeah, you'll probably want to do that. What you propose is certainly not simple, nor will it be quick (regardless of your programming talent). Go for it. DIY-PROM'ers love innovation! :-D
#15
TGO Supporter
Join Date: Jan 2000
Location: In your ear. No, the other one.
Posts: 1,861
Likes: 0
Received 0 Likes
on
0 Posts
Car: '89 Trans Am WS6
Engine: 350 TPI
Transmission: T5WC
Axle/Gears: 3.08 posi
Feel free to ask (me) if you need help. I've been known to write a line or two of code (some even directly related to what you're talking about).
#16
Junior Member
Join Date: Oct 2004
Location: Austin, TX
Posts: 17
Likes: 0
Received 0 Likes
on
0 Posts
Originally posted by Mangus
Feel free to ask (me) if you need help. I've been known to write a line or two of code (some even directly related to what you're talking about).
Feel free to ask (me) if you need help. I've been known to write a line or two of code (some even directly related to what you're talking about).
And to think, I got started on this idea about 6 months ago when a buddy of mine said off hand "carburetors suck off road, you should adapt a GM TBI system to your engine"...
#17
Senior Member
Join Date: Oct 1999
Posts: 6,621
Likes: 0
Received 1 Like
on
1 Post
Car: 91 Red Sled
Axle/Gears: 10bolt Richmond 3.73 Torsen
<-- been working on Autotune for over a year on and off. I've got all the code done for the fueling and am now working on the romulator integration side. Once I put the code together I should have an autotune up and running in real-time. It's pretty advanced if I might say so myself. There is code that will calculate your "normal engine temp" and do all of the CTS, IAT, voltage, etc. compensation tables. I've even got AE code but that hasn't been tested yet on the bench. I've already got a good plan for the AE code and it shouldn't be too difficult with lockers... impossible with the c3 ALDL .
Once the fueling works and checks out okay I'll move into the spark advance and economy code. Right now I only have 1 EGT and 1 CHT sensor. I'd like to get at least 4 of each for obvious reasons. The spark stuff is tricky and I haven't a fool proof plan on how to code it yet. Stage 1 would probably be just looking at the knock counters but I fear that won't be enough. The fuel economy will be difficult because I don't have a fuel flow meter, I'll have to come up with a "value" that will be derived from MAP, pulse width, CHT, EGT, and AFR. Too bad I couldn't find a TBI flow bencher that would clean my injectors let alone run them at varied pressures and pulse widths to chart fuel flow .
I suggest you guys do your own code from scratch. I'm riding solo on this one since I seem to be alone with the lockers and TBI. The slower ALDL rate will limit you to nothing more than a smoother closed loop . If the speedreader comes out I'm sure I could adapt my software to doing the complicated stuff .
Once the fueling works and checks out okay I'll move into the spark advance and economy code. Right now I only have 1 EGT and 1 CHT sensor. I'd like to get at least 4 of each for obvious reasons. The spark stuff is tricky and I haven't a fool proof plan on how to code it yet. Stage 1 would probably be just looking at the knock counters but I fear that won't be enough. The fuel economy will be difficult because I don't have a fuel flow meter, I'll have to come up with a "value" that will be derived from MAP, pulse width, CHT, EGT, and AFR. Too bad I couldn't find a TBI flow bencher that would clean my injectors let alone run them at varied pressures and pulse widths to chart fuel flow .
I suggest you guys do your own code from scratch. I'm riding solo on this one since I seem to be alone with the lockers and TBI. The slower ALDL rate will limit you to nothing more than a smoother closed loop . If the speedreader comes out I'm sure I could adapt my software to doing the complicated stuff .
#18
Senior Member
Thread Starter
Join Date: Nov 2003
Location: Rockport, TX
Posts: 647
Likes: 0
Received 0 Likes
on
0 Posts
Car: 1980 Jeep CJ7
Engine: AMC 360 with GM TBI
Transmission: TF727 set for kill...let's get it on!!!
derf, remember I have an AutoPROM and we can use that for the interface. I also have all those ALDL defs that Mangus linked to in his post. If you do get that blog forum or email list up and running, let me know and I'll get in on the fray too.
#19
Junior Member
Join Date: Oct 2004
Location: Austin, TX
Posts: 17
Likes: 0
Received 0 Likes
on
0 Posts
Originally posted by jeepguy553
derf, remember I have an AutoPROM and we can use that for the interface. I also have all those ALDL defs that Mangus linked to in his post. If you do get that blog forum or email list up and running, let me know and I'll get in on the fray too.
derf, remember I have an AutoPROM and we can use that for the interface. I also have all those ALDL defs that Mangus linked to in his post. If you do get that blog forum or email list up and running, let me know and I'll get in on the fray too.
#21
Junior Member
Join Date: Apr 2004
Location: College Station, Tx
Posts: 77
Likes: 0
Received 0 Likes
on
0 Posts
Car: Land Rovers
Engine: 3950cc
Transmission: some autos, some manuals
Originally posted by derf
I'm actually a fan of designing my own operating system. I've done that a few times for custom embedded avionics systems. They weren't complex but they sure did do real time better than any general purpose OS I ever used.
I've been using Linux since kernel 1.2.7 off and on. I know a lot of DIY tuners are also Linux geeks so the program has to be available for them. However, a Windoze version would appeal to a broad segment of the population too. I've done cross platform development (NT and Linux) for device drivers before so this shouldn't be too much of a challenge, especially if we design it to be modular. It's not too hard to swap out the comm modules and the calls to create and run the window interface if it's designed to do that from the begining.
Cool. I'll dig into that for a while.
I'm actually a fan of designing my own operating system. I've done that a few times for custom embedded avionics systems. They weren't complex but they sure did do real time better than any general purpose OS I ever used.
I've been using Linux since kernel 1.2.7 off and on. I know a lot of DIY tuners are also Linux geeks so the program has to be available for them. However, a Windoze version would appeal to a broad segment of the population too. I've done cross platform development (NT and Linux) for device drivers before so this shouldn't be too much of a challenge, especially if we design it to be modular. It's not too hard to swap out the comm modules and the calls to create and run the window interface if it's designed to do that from the begining.
Cool. I'll dig into that for a while.
I know this is an old thread that was recently resurected, but this link helped me figure out how to read 160 baud at higher serial port rates.
http://www.techedge.com.au/vehicle/a.../160serial.htm
Last edited by robertf; 07-02-2005 at 05:46 PM.
#22
TGO Supporter
Join Date: Jan 2000
Location: In your ear. No, the other one.
Posts: 1,861
Likes: 0
Received 0 Likes
on
0 Posts
Car: '89 Trans Am WS6
Engine: 350 TPI
Transmission: T5WC
Axle/Gears: 3.08 posi
Originally posted by robertf
thats why Java is your friend
I know this is an old thread that was recently resurected, but this link helped me figure out how to read 160 baud at higher serial port rates.
http://www.techedge.com.au/vehicle/a.../160serial.htm
thats why Java is your friend
I know this is an old thread that was recently resurected, but this link helped me figure out how to read 160 baud at higher serial port rates.
http://www.techedge.com.au/vehicle/a.../160serial.htm
#23
Junior Member
Join Date: Apr 2004
Location: College Station, Tx
Posts: 77
Likes: 0
Received 0 Likes
on
0 Posts
Car: Land Rovers
Engine: 3950cc
Transmission: some autos, some manuals
Originally posted by Mangus
How do you read 160 baud at higher rates? The ECM will only output at 160 baud. You'd have to speed up the ECM's output before you can read anything any faster.
How do you read 160 baud at higher rates? The ECM will only output at 160 baud. You'd have to speed up the ECM's output before you can read anything any faster.
to get it to work you sample the data at a higher speed, (2400,9600,etc), once 8 bits are recieved at the higher speed(which is less than the time it takes for the 160 baud stream to finish its stop bit) it waits for the stop bit and start bit from the 160 baud stream.
Then you look at the timing of the 160 baud stream vs the higher rate and at whichever moment the ecm bit was sent you read the bit that corresponds to that moment from your sampled 9600 baud byte and its your 1 or 0
Last edited by robertf; 07-02-2005 at 07:42 PM.
#24
TGO Supporter
Join Date: Jan 2000
Location: In your ear. No, the other one.
Posts: 1,861
Likes: 0
Received 0 Likes
on
0 Posts
Car: '89 Trans Am WS6
Engine: 350 TPI
Transmission: T5WC
Axle/Gears: 3.08 posi
Oh, yes, you need to read at 2400 or 4800 (for USB). Sorry, I misunderstood what you were trying to say.
#26
Member
Join Date: Jun 2001
Location: Eh?
Posts: 391
Likes: 0
Received 0 Likes
on
0 Posts
Car: 1988 Monte Carlo SS
Engine: 5.7L TPI
Transmission: T5
Axle/Gears: 3.73
Originally Posted by derf
The 8 bit controllers on some of these ECMs aren't really fast enough to do that level of calibration. Besides, they don't really have a way to update their own PROM tables without serious hacks.
Thread
Thread Starter
Forum
Replies
Last Post