Project Journals > Steve

End of an era

<< < (2/2)

MadScientist267:
Ok well here goes... Part 2 of it anyway.

The initial comb thru the file system left me feeling a little like an alien in my own world... But once I scraped the rust off... I was able to piece together the last time slice worth of data at least, as the graphs need still need to be reconstructed from the data structure that they're built from.

First and foremost, the theory that the classic had already shut down and imminent doom had set in before the Pi dropped offline has been disproven at this point.

It was shown to have been still responding to polls, accurately time stamped, based on data from the last parachute backup: Dec 12 at 8AM.

This data set may precede the actual protective disconnect by up to 59 minutes before it's final breath...

1. Battery voltage reported by the classic matched with readings from the battery monitor's ADC

2. Classic had a PCB temp well above ambient temp

3. No polling errors related to the classic in the logs



Battery monitor system reports:

Pair A:
Block #1: 5.260V
Block #2: 3.156V
Total: 8.416V

Pair B:
Block #3: 5.442V
Block #4: 2.956V
Total: 8.398V


Classic reports:

Mode state reported was: Resting

Battery voltage reported was: 8.3V.

Net current reported by WBJR was: -2.9A

8.3V at 2.9A indicates a load of about 24W. That's about right with the intake fan running and everything else in deep sleep mode for minimal system operation, with little or no solar input.

It was normal to see the classic report slightly under the battery monitor under load conditions (and slightly higher during charging)

PCB temp reported was: 17.9C
FET temp reported was: 8.4C

FET temp being below board temp by this much also indicates very little if any production by the PV, consistent with the time of day and expected negative net current being drawn from the battery.


Pyranometer report:

The pyro shows it was receiving light, adding a bit of insult to injury.

Pyro reported insolation: 27.675 W/m²

It would appear that by 8AM, the snow was gone at least off of the sensor. This doesn't necessarily indicate the entire roof was clear but at least *some* light would have been able to theoretically reach at least some of the PV. Exact sky conditions at that moment are still unknown at this point.


Temperatures reported:

Ambient outside temp reported: 7C
Ambient inside temp reported: 1.625C
Thermal Mass discharge temp: 0.75C

Temperatures had at least become favorable for melt outside, even if there was no direct light to assist this. Going by the lag in both inside ambient and thermal mass probes, conditions just prior to this time slice were more conducive to maintaining the freeze than helping with melt.


Battery Health:

SoC reported by classic was: 45%

With the voltages as low as they had become, this indicates significant degradation to the batteries. They really HAD taken a beating

I knew they were in rough shape but didn't expect they'd been reduced to only about half of label... Not exactly surprised, just didn't estimate things had progressed that far.

I figure a fair chunk of it probably happened in the last month prior to being able to move it to get better light... it was sitting mostly shaded. It's of course difficult to say how much damage was done when, as they served a hard life in normal operation, in addition to the extra abuse at the end caused by the shade.


A little bit of feel good story to tack on to the end... Sorta...

top - 13:30:50 up 367 days

So between 8:30 and 8:35AM... The Pi's power supply made a clean break.

(Side note - The RTC scheme used had the classic operating as the master clock, which was set to GMT. The Pi used this time as well, the corrections were made for local in the graphs on the fly)

That really sucks... Couple more hours and I figure more than likely it would have actually survived. For how much longer, well of course, anyone's guess.

It also could have been a bit worse...

One more tidbit I caught just before wrapping up this part of the investigation...

So used to seeing "meaningless" crap in the log tail that I didn't scrutinize it closely at first...

The final straw for the Pi may have been the actual alarm itself tipping it's power that last little bit...

I show 3 recorded poll failures from the ADC that pissed python off, probably because the regulators on the ADC couldn't hold 5V anymore

So for about 15 minutes before the shutdown, the alarm LED was pulsing... The +5V feed for the LED (1W red) being taken from the Pi's header

The supply feeding the Pi was no slouch, a full blown micro PC DC-DC supply complete with crowbar... There was a reason for that, even tho this wasn't exactly one of the scenarios envisioned when it was selected.

I'm just glad it worked out as "designed". I fear with great certainty that in this situation, a simple buck converter supply would have likely wrecked shop because it wouldn't have shut itself down in paranoia and the pi could have fringe cycled it's way into corrupting the SD card. Granted it was designed at the software level to do minimal writing... But that's only on my end. The OS was standard Raspbian Wheezy, which I know to a point is tweaked for SD but I don't know to what extent, and hadn't really done any tweaking of my own on it for those purposes.

Either way, all the data appears to be intact... It's just not in a very friendly form to externally recover and directly view.

The next fun part is setting up a dummy run of ranges with the date/time forced so it generates the graphs as were at time of death... Then I can post the pretty pictures of what only I can "see" at the moment in my mind, as well as the entire "bigger picture".

There's no doubt that at some point things just had gone too far and the classic gave up, as suspected. The order this took place in however, a bit of a surprise. If I'm right, there won't be any "flat" lines on the data points that come from the classic. It also means that nobody will ever know exactly when the classic gave up and threw the batteries, once and for good, up under the... Truck.

Guess we'll see, won't we. I'll get to it as soon as I can, there's a lot going on at the moment that takes precedence.

Till then...

rossw:
Nice work getting all that after the event, Steve.
Doesn't change things of course, but I'm sure you'll have some amount of pride in knowing you worked out how it all went down.

Navigation

[0] Message Index

[*] Previous page