You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-21 - 08:45
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Fri Mar 21, 2025 00:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
217462200183,12sleep10-21swapper/119:05:071
238582195178,12sleep00-21swapper/019:08:350
239532183167,11sleep30-21swapper/319:09:513
239222179163,12sleep20-21swapper/219:09:252
276612430,0sleep10-21swapper/122:06:031
260642420,0sleep10-21swapper/121:48:531
41962360,1sleep22415799cyclictest22:44:272
35102130,0sleep20-21swapper/222:27:372
302062100,0sleep10-21swapper/100:06:331
267512100,0sleep30-21swapper/322:05:283
31418290,0sleep331420-21processes19:20:183
241539991,4cyclictest9624-21munin-run21:40:010
241579982,2cyclictest31-21ksoftirqd/223:10:102
241579982,2cyclictest31-21ksoftirqd/219:20:002
241579981,1cyclictest31-21ksoftirqd/200:20:202
241539980,7cyclictest0-21swapper/020:15:090
241579973,2cyclictest31-21ksoftirqd/221:10:012
241579972,2cyclictest31-21ksoftirqd/223:55:162
241579972,1cyclictest31-21ksoftirqd/219:55:212
241579972,1cyclictest26489-21apt-get21:15:142
241579971,2cyclictest31-21ksoftirqd/222:50:082
241579971,2cyclictest31-21ksoftirqd/222:35:142
241579971,2cyclictest31-21ksoftirqd/220:05:132
241579971,2cyclictest31-21ksoftirqd/219:10:212
241579971,1cyclictest31-21ksoftirqd/223:00:142
241579970,6cyclictest11049-21df_inode20:55:132
241579970,1cyclictest31-21ksoftirqd/223:20:122
241539973,4cyclictest27522-21munin-run22:40:000
29948260,0sleep10-21swapper/119:20:001
241639960,5cyclictest16920-21ssh21:10:103
241639960,5cyclictest0-21swapper/321:30:123
241579966,0cyclictest301ktimersoftd/223:35:182
241579964,1cyclictest31-21ksoftirqd/221:55:152
241579964,0cyclictest31-21ksoftirqd/223:50:092
241579964,0cyclictest31-21ksoftirqd/220:10:172
241579964,0cyclictest31-21ksoftirqd/219:30:002
241579964,0cyclictest31-21ksoftirqd/200:04:262
241579963,1cyclictest31-21ksoftirqd/221:50:182
241579962,2cyclictest31-21ksoftirqd/220:45:162
241579962,1cyclictest31-21ksoftirqd/222:30:102
241579961,2cyclictest31-21ksoftirqd/223:17:522
241579961,1cyclictest81rcu_preempt00:30:102
241579961,1cyclictest0-21swapper/220:00:122
241579960,2cyclictest31-21ksoftirqd/222:55:142
241579960,2cyclictest31-21ksoftirqd/222:20:142
241579960,2cyclictest31-21ksoftirqd/221:35:222
241579960,2cyclictest31-21ksoftirqd/219:40:142
241579960,1cyclictest81rcu_preempt23:40:122
241579960,1cyclictest31-21ksoftirqd/222:15:142
241579960,1cyclictest31-21ksoftirqd/222:10:082
241579960,1cyclictest31-21ksoftirqd/221:25:012
241579960,1cyclictest31-21ksoftirqd/220:30:132
241579960,1cyclictest31-21ksoftirqd/219:55:012
241579960,1cyclictest31-21ksoftirqd/219:25:012
241579960,1cyclictest31-21ksoftirqd/200:35:152
241579960,1cyclictest31-21ksoftirqd/200:30:002
241579960,1cyclictest31-21ksoftirqd/200:10:122
241539960,6cyclictest0-21swapper/023:40:160
241639955,0cyclictest27458-21kworker/3:023:46:243
241639955,0cyclictest27458-21kworker/3:023:23:593
241639955,0cyclictest27458-21kworker/3:022:39:133
241639955,0cyclictest27458-21kworker/3:020:13:413
241639952,3cyclictest30997-21snmpd00:21:373
241639950,5cyclictest0-21swapper/300:16:163
241639950,4cyclictest0-21swapper/319:55:133
241639950,0cyclictest0-21swapper/321:15:253
241579954,0cyclictest31-21ksoftirqd/222:00:012
241579954,0cyclictest31-21ksoftirqd/221:40:222
241579954,0cyclictest31-21ksoftirqd/221:40:222
241579953,2cyclictest301ktimersoftd/222:07:512
241579953,0cyclictest31-21ksoftirqd/223:30:082
241579953,0cyclictest31-21ksoftirqd/220:15:012
241579952,0cyclictest81rcu_preempt21:25:382
241579951,2cyclictest31-21ksoftirqd/221:30:112
241579951,1cyclictest31-21ksoftirqd/223:07:212
241579951,1cyclictest31-21ksoftirqd/220:41:052
241579951,1cyclictest31-21ksoftirqd/219:35:172
241579951,0cyclictest81rcu_preempt21:05:012
241579951,0cyclictest31-21ksoftirqd/221:46:442
241579950,2cyclictest3335-21gltestperf19:30:172
241579950,2cyclictest31-21ksoftirqd/222:47:212
241579950,2cyclictest12604-21kworker/2:220:35:182
241579950,1cyclictest9837-21sed20:50:242
241579950,1cyclictest31-21ksoftirqd/223:45:052
241579950,1cyclictest31-21ksoftirqd/221:10:102
241579950,1cyclictest29434-21cut20:25:172
241579950,1cyclictest13749-21aten2_r5power_c00:15:142
241579950,1cyclictest0-21swapper/220:20:112
241579950,0cyclictest81rcu_preempt00:05:082
241579950,0cyclictest0-21swapper/223:26:062
241549950,5cyclictest0-21swapper/122:54:371
241549950,5cyclictest0-21swapper/122:20:161
241549950,5cyclictest0-21swapper/121:20:381
241549950,5cyclictest0-21swapper/100:35:441
241539955,0cyclictest3-21ksoftirqd/021:05:020
241539955,0cyclictest0-21swapper/021:24:400
241539954,1cyclictest3-21ksoftirqd/023:35:150
241539951,3cyclictest30997-21snmpd21:09:330
241539951,3cyclictest30997-21snmpd20:22:590
241539951,2cyclictest0-21swapper/021:15:140
241539950,5cyclictest0-21swapper/023:20:110
241539950,5cyclictest0-21swapper/000:15:140
241539950,4cyclictest24675-21irqstats00:20:150
241539950,4cyclictest15726-21apt-get20:00:130
241539950,4cyclictest0-21swapper/019:30:140
241539950,4cyclictest0-21swapper/000:09:580
241539950,0cyclictest0-21swapper/021:45:020
27068240,0sleep10-21swapper/100:21:141
241639944,0cyclictest27458-21kworker/3:023:01:373
241639944,0cyclictest27458-21kworker/3:021:09:403
241639944,0cyclictest27458-21kworker/3:020:58:273
241639944,0cyclictest27458-21kworker/3:020:36:043
241639944,0cyclictest27458-21kworker/3:020:24:533
241639944,0cyclictest27458-21kworker/3:020:02:293
241639944,0cyclictest27458-21kworker/3:019:28:533
241639944,0cyclictest0-21swapper/323:28:063
241639941,3cyclictest9465-21needreboot23:55:163
241639941,3cyclictest25880-21cat23:30:113
241639940,4cyclictest7797-21ssh00:11:373
241639940,4cyclictest7099-21python20:45:183
241639940,4cyclictest28555-21diskstats22:40:113
241639940,4cyclictest15883-21diskmemload23:10:073
241639940,4cyclictest0-21swapper/323:50:013
241639940,4cyclictest0-21swapper/323:40:123
241639940,4cyclictest0-21swapper/323:35:123
241639940,4cyclictest0-21swapper/323:07:573
241639940,4cyclictest0-21swapper/322:56:183
241639940,4cyclictest0-21swapper/322:53:273
241639940,4cyclictest0-21swapper/322:45:143
241639940,4cyclictest0-21swapper/322:24:323
241639940,4cyclictest0-21swapper/322:15:123
241639940,4cyclictest0-21swapper/322:01:273
241639940,4cyclictest0-21swapper/321:46:473
241639940,4cyclictest0-21swapper/321:40:133
241639940,4cyclictest0-21swapper/321:40:133
241639940,4cyclictest0-21swapper/321:35:143
241639940,4cyclictest0-21swapper/321:25:233
241639940,4cyclictest0-21swapper/321:21:133
241639940,4cyclictest0-21swapper/300:06:313
241639940,4cyclictest0-21swapper/300:00:133
241639940,3cyclictest9888-21sed19:45:013
241639940,3cyclictest7707-21cat22:30:003
241639940,3cyclictest7208-21latency_hist19:40:003
241639940,3cyclictest6583-21sh22:29:353
241639940,3cyclictest4031-21ssh21:54:073
241639940,3cyclictest2505-21diskstats22:10:103
241639940,3cyclictest18912-21ssh00:35:013
241639940,3cyclictest15883-21diskmemload23:15:013
241579942,0cyclictest31-21ksoftirqd/219:47:052
241549944,0cyclictest0-21swapper/123:43:391
241549944,0cyclictest0-21swapper/123:28:041
241549944,0cyclictest0-21swapper/123:09:111
241549944,0cyclictest0-21swapper/122:42:231
241549944,0cyclictest0-21swapper/121:57:261
241549944,0cyclictest0-21swapper/121:51:541
241549944,0cyclictest0-21swapper/121:25:081
241549944,0cyclictest0-21swapper/100:03:261
241549942,1cyclictest0-21swapper/122:10:081
241549942,1cyclictest0-21swapper/100:15:141
241549941,2cyclictest0-21swapper/100:10:071
241549940,4cyclictest0-21swapper/123:37:061
241549940,0cyclictest0-21swapper/123:51:041
241549940,0cyclictest0-21swapper/122:35:501
241549940,0cyclictest0-21swapper/122:17:041
241539944,0cyclictest0-21swapper/022:31:010
241539942,1cyclictest30997-21snmpd20:41:290
241539942,1cyclictest30997-21snmpd20:29:080
241539942,1cyclictest30997-21snmpd20:06:270
241539942,1cyclictest30997-21snmpd19:48:320
241539942,1cyclictest30997-21snmpd19:19:410
241539942,1cyclictest15274-21latency_hist20:00:000
241539940,4cyclictest31126-21df_inode23:50:080
241539940,4cyclictest0-21swapper/023:55:110
241539940,4cyclictest0-21swapper/023:30:080
241539940,4cyclictest0-21swapper/023:00:140
241539940,4cyclictest0-21swapper/022:09:570
241539940,4cyclictest0-21swapper/021:50:150
241539940,4cyclictest0-21swapper/020:10:150
241539940,4cyclictest0-21swapper/019:27:070
241539940,3cyclictest9161-21cat19:40:200
241539940,3cyclictest8141-21users21:55:230
241539940,3cyclictest7104-21ssh22:46:080
241539940,3cyclictest6153-21ssh00:28:240
241539940,3cyclictest6111-21diskstats20:45:120
241539940,3cyclictest3566-21ssh00:10:060
241539940,3cyclictest32299-21sed19:25:000
241539940,3cyclictest30997-21snmpd23:17:050
241539940,3cyclictest2838-21hddtemp_smartct22:10:110
241539940,3cyclictest26845-21ssh22:55:590
241539940,3cyclictest24435-21ssh21:14:060
241539940,3cyclictest24304-21ssh23:29:540
241539940,3cyclictest22560-21ssh22:03:070
241539940,3cyclictest22436-21aten2_r5power_c21:30:110
241539940,3cyclictest21997-21ssh22:20:090
241539940,3cyclictest21513-21ssh00:35:300
241539940,3cyclictest20195-21expr23:10:100
241539940,3cyclictest18483-21/usr/sbin/munin00:00:140
241539940,3cyclictest15883-21diskmemload22:40:010
241539940,3cyclictest15850-21rm00:33:140
241539940,3cyclictest15832-21ssh21:42:370
241539940,3cyclictest15832-21ssh21:42:370
241539940,3cyclictest14800-21ssh23:07:250
241539940,3cyclictest14085-21ssh21:25:070
241539940,3cyclictest12619-21df22:15:140
241539940,3cyclictest0-21swapper/022:52:130
241539940,3cyclictest0-21swapper/020:35:110
241639932,1cyclictest8058-21apt-get20:50:163
241639932,1cyclictest19134-21if_eth020:05:163
241639931,1cyclictest0-21swapper/320:15:093
241639930,3cyclictest9688-21sh00:30:113
241639930,3cyclictest6027-21apt21:55:153
241639930,3cyclictest15883-21diskmemload00:25:203
241639930,3cyclictest0-21swapper/321:00:133
241639930,3cyclictest0-21swapper/320:40:063
241639930,3cyclictest0-21swapper/320:30:133
241639930,3cyclictest0-21swapper/320:25:053
241639930,3cyclictest0-21swapper/319:50:083
241639930,3cyclictest0-21swapper/319:45:513
241639930,3cyclictest0-21swapper/319:30:143
241639930,3cyclictest0-21swapper/319:15:123
241639930,3cyclictest0-21swapper/319:10:133
241549933,0cyclictest0-21swapper/123:56:011
241549933,0cyclictest0-21swapper/123:15:421
241549932,1cyclictest0-21swapper/119:40:111
241549931,1cyclictest0-21swapper/123:50:001
241549931,1cyclictest0-21swapper/123:20:111
241549931,1cyclictest0-21swapper/123:10:071
241549931,1cyclictest0-21swapper/122:05:011
241549931,1cyclictest0-21swapper/121:30:111
241549931,1cyclictest0-21swapper/121:15:141
241549931,1cyclictest0-21swapper/121:10:081
241549931,1cyclictest0-21swapper/120:15:091
241549931,1cyclictest0-21swapper/119:55:131
241549931,1cyclictest0-21swapper/119:20:121
241549930,3cyclictest0-21swapper/119:10:581
241549930,2cyclictest20223-21chrt20:07:321
241549930,2cyclictest0-21swapper/121:40:141
241549930,2cyclictest0-21swapper/121:40:131
241549930,2cyclictest0-21swapper/120:25:101
241539931,1cyclictest31141-21grep20:30:150
241539931,1cyclictest30997-21snmpd23:45:010
241539930,3cyclictest31799-21rm22:25:010
241539930,3cyclictest0-21swapper/020:55:110
241539930,3cyclictest0-21swapper/019:50:080
241539930,3cyclictest0-21swapper/019:35:180
241539930,3cyclictest0-21swapper/019:10:070
241539930,2cyclictest7804-21cron20:50:000
241549922,0cyclictest0-21swapper/119:46:011
241549921,1cyclictest156682chrt21:07:341
241549921,1cyclictest123982chrt20:55:321
241549921,1cyclictest0-21swapper/122:30:011
241549921,1cyclictest0-21swapper/120:40:121
241549921,1cyclictest0-21swapper/120:35:111
241549921,1cyclictest0-21swapper/120:10:151
241549921,1cyclictest0-21swapper/119:25:131
241549920,1cyclictest6416-21chrt22:45:421
241549920,1cyclictest3909-21chrt22:27:521
241549920,1cyclictest1929-21taskset00:25:421
241549920,1cyclictest0-21swapper/123:30:081
241549920,1cyclictest0-21swapper/123:00:141
241549920,1cyclictest0-21swapper/122:55:021
241549920,1cyclictest0-21swapper/121:35:021
241549920,1cyclictest0-21swapper/121:00:131
241549920,1cyclictest0-21swapper/120:45:111
241549920,1cyclictest0-21swapper/120:30:131
241549920,1cyclictest0-21swapper/120:20:111
241549920,1cyclictest0-21swapper/120:00:011
241549920,1cyclictest0-21swapper/119:50:081
241549920,1cyclictest0-21swapper/119:35:141
241549920,1cyclictest0-21swapper/119:30:151
241549920,1cyclictest0-21swapper/100:30:101
241549910,1cyclictest0-21swapper/120:50:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional