In the data frame of probabilities over time return first column name where value is < .5 for each row

Question

Given a pandas data frame like the following where the column names are the time, the rows are each of the subjects, and the values are probabilities return the column name (or time) the first time the probability is less than .50 for each subject in the data frame. The probabilities are always descending from 1-0 I. have tried looping though the data frame but it is not computationally efficient.

subject id	0	1	2	3	4	5	6	7	…	669	670	671
1	1	0.997913	0.993116	0.989017	0.976157	0.973078	0.968056	0.963685	…	0.156092	0.156092	0.156092
2	1	0.990335	0.988685	0.983145	0.964912	0.958	0.952	0.946995	…	0.148434	0.148434	0.148434
3	1	0.996231	0.990571	0.985775	0.976809	0.972736	0.969633	0.966116	…	0.17037	0.17037	0.17037
4	1	0.997129	0.994417	0.991054	0.978795	0.974216	0.96806	0.963039	…	0.15192	0.15192	0.15192
5	1	0.997728	0.993598	0.986641	0.98246	0.977371	0.972874	0.96816	…	0.154545	0.154545	0.154545
6	1	0.998134	0.995564	0.989901	0.986941	0.982313	0.972951	0.969645	…	0.17473	0.17473	0.17473
7	1	0.995681	0.994131	0.990401	0.974494	0.967941	0.961859	0.956636	…	0.144753	0.144753	0.144753
8	1	0.997541	0.994904	0.991941	0.983389	0.979375	0.973158	0.966358	…	0.158763	0.158763	0.158763
9	1	0.992253	0.989064	0.979258	0.955747	0.948842	0.942899	0.935784	…	0.150291	0.150291	0.150291

Goal Output

subject id	time prob < .05
1	100
2	99
3	34
4	19
5	600
6	500
7	222
8	111
9	332

Solution 1

Since the probabilities are always descending you can do this:

>>> df.set_index("subject id").gt(.98).sum(1)
subject id
1    4
2    4
3    4
4    4
5    5
6    6
7    4
8    5
9    3
dtype: int64

note: I'm using .98 instead of .5 because I'm using only a portion of the data.

---

Data used

{'subject id': {0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8, 8: 9},
 '0': {0: 1, 1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1, 7: 1, 8: 1},
 '1': {0: 0.997913,
  1: 0.990335,
  2: 0.996231,
  3: 0.997129,
  4: 0.997728,
  5: 0.998134,
  6: 0.995681,
  7: 0.997541,
  8: 0.992253},
 '2': {0: 0.993116,
  1: 0.988685,
  2: 0.990571,
  3: 0.994417,
  4: 0.993598,
  5: 0.995564,
  6: 0.994131,
  7: 0.994904,
  8: 0.989064},
 '3': {0: 0.989017,
  1: 0.983145,
  2: 0.985775,
  3: 0.991054,
  4: 0.986641,
  5: 0.989901,
  6: 0.990401,
  7: 0.991941,
  8: 0.979258},
 '4': {0: 0.976157,
  1: 0.964912,
  2: 0.976809,
  3: 0.978795,
  4: 0.98246,
  5: 0.986941,
  6: 0.974494,
  7: 0.983389,
  8: 0.955747},
 '5': {0: 0.973078,
  1: 0.958,
  2: 0.972736,
  3: 0.974216,
  4: 0.977371,
  5: 0.982313,
  6: 0.967941,
  7: 0.979375,
  8: 0.948842},
 '6': {0: 0.968056,
  1: 0.952,
  2: 0.969633,
  3: 0.96806,
  4: 0.972874,
  5: 0.972951,
  6: 0.961859,
  7: 0.973158,
  8: 0.942899},
 '7': {0: 0.963685,
  1: 0.946995,
  2: 0.966116,
  3: 0.963039,
  4: 0.96816,
  5: 0.969645,
  6: 0.956636,
  7: 0.966358,
  8: 0.935784}}

Solution 2

If I understand correctly, I think this is what you are looking for:

df.where(df.lt(.5)).idxmax(axis=1)

Comments

[Admin]

Can you post the list of items for your DD-WRT host?

[Admin]

@StephenKing This is the item list in my DD-WRT host, which it have 122 items. (imageshack.us/a/img703/7182/azsu.png) (imageshack.us/a/img703/7182/azsu.png) (imageshack.us/a/img844/7237/2zdn.png)

[Admin]

I did research on the Zabbix Agent on DD-WRT seem the atheros chipset doesn't support the Zabbix Agent, can I using the default SNMP function in DD-WRT to monitoring by using the Zabbix?

[Admin]

Where did you find that information? Did you try to compile the agent from source? Here's a tutorial: diegolima.org/wordpress/?p=407

[Admin]

And yes.. that's the second choice I mentioned: Add the Host in Zabbix using the Template_SNMPv2_Device. You can also add additional SNMP items that are exposed by the router's MIB, but not in the Template.

[Jien Wai]

What I did rightnow is my currently my router IP is 192.168.1.1 and I create a host at Zabbix also put the SNMP interfaces as 191.168.1.1. I were follow this http://projectdaenney.org/blog/2012/10/14/zabbix-baby-steps-‌​part-1/ to configure the setting. After I done this basic setting, I create a host to monitoring my router which I have add the Template SNMP Generic as my template., but I still unable to monitoring my router which is come wiht Timeout while connecting to "192.168.1.1:161`.

[StephenKing]

Just to make sure: You've activated the SNMP agent and can also connect to it through an SNMP browser on your PC? Community names (public/private) match? What's not clear to me is "I create a host... After I done this basic setting, I create a host.." - you're not creating two hosts, aren't you?

[Pablo C]

@user14473305 I'm happy to help! :D