You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 13:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Wed Dec 03, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13416991024,51cyclictest0-21swapper/023:59:060
1341699994,51cyclictest0-21swapper/022:59:060
1341699994,51cyclictest0-21swapper/022:59:060
13420999337,49cyclictest14713-21cat19:58:553
1342099864,13cyclictest0-21swapper/323:26:193
1342099853,32cyclictest0-21swapper/321:53:503
1342099853,12cyclictest0-21swapper/321:54:243
13420998425,10cyclictest876-21uniq19:29:153
13420998418,9cyclictest15182-21unixbench_multi23:19:203
1342099834,13cyclictest0-21swapper/319:09:273
13420998323,10cyclictest1369-21runrttasks23:49:013
13420998225,10cyclictest25759-21cpuspeed_turbos23:44:013
13420998117,11cyclictest0-21swapper/322:31:273
1342099804,22cyclictest0-21swapper/320:20:503
13420998036,39cyclictest0-21swapper/320:39:133
13420998026,44cyclictest0-21swapper/319:08:453
13420998024,9cyclictest5692-21fschecks_count00:09:023
13420998021,10cyclictest12600-21cpuspeed_turbos22:09:033
13420997935,40cyclictest0-21swapper/322:37:503
13420997931,7cyclictest1397-21munin-node22:54:043
13420997924,8cyclictest31778-21cpuspeed_turbos22:49:023
13420997923,10cyclictest27897-21aten_r1power_cu21:34:023
13420997923,10cyclictest27897-21aten_r1power_cu21:34:023
13420997922,11cyclictest0-21swapper/323:04:183
13420997918,9cyclictest18325-21munin-node21:14:213
1342099784,36cyclictest0-21swapper/320:57:063
1342099784,36cyclictest0-21swapper/320:57:063
13420997829,8cyclictest1369-21runrttasks18:50:463
13420997829,8cyclictest1369-21runrttasks18:50:463
13420997821,8cyclictest14705-21munin-node22:14:243
13420997821,8cyclictest14705-21munin-node22:14:243
1342099782,35cyclictest0-21swapper/322:44:003
13420997728,8cyclictest12515-21sensors_temp23:14:183
13420997727,7cyclictest31014-21cut19:24:133
13420997723,7cyclictest32308-21seq23:58:133
13420997721,8cyclictest1243-21snmpd21:00:573
13420997717,10cyclictest9481-21sshd19:45:133
13420997625,7cyclictest17171-21munin-node20:04:163
13420997624,8cyclictest15942-21munin-node21:09:223
13420997617,10cyclictest27957-21munin-node19:19:313
1342099745,14cyclictest0-21swapper/320:28:543
13420997430,8cyclictest19038-21gltestperf23:29:083
1342099743,15cyclictest0-21swapper/321:19:133
13420997424,9cyclictest4629-21vmstat19:34:303
13420997417,12cyclictest0-21swapper/320:13:123
1342099734,15cyclictest0-21swapper/323:09:183
13420997332,35cyclictest0-21swapper/323:35:013
1342099733,33cyclictest16051-21telnet18:54:093
13420997322,7cyclictest1243-21snmpd19:50:453
1341999733,63cyclictest0-21swapper/220:38:122
13420997234,7cyclictest244982sleep321:24:203
13420997229,10cyclictest25668-21telnet19:14:103
1342099705,15cyclictest0-21swapper/322:08:503
13420997023,8cyclictest7350-21yum-updatesd-he19:43:443
13420996930,10cyclictest0-21swapper/322:59:153
13420996930,10cyclictest0-21swapper/322:59:153
13420996927,9cyclictest0-21swapper/319:59:193
1342099683,14cyclictest0-21swapper/323:39:083
1342099683,14cyclictest0-21swapper/323:39:083
13420996826,10cyclictest0-21swapper/322:24:443
13420996824,9cyclictest0-21swapper/321:39:073
13420996821,8cyclictest2967-21cpuspeed_turbos00:04:003
13416996845,10cyclictest0-21swapper/019:44:080
1342099674,15cyclictest0-21swapper/321:44:203
13420996725,10cyclictest0-21swapper/322:23:493
13420996720,9cyclictest29166-21telnet20:29:073
1341999675,55cyclictest0-21swapper/219:54:122
237002650,6sleep023696-21cut21:24:090
1341899654,53cyclictest0-21swapper/119:24:211
1341699653,56cyclictest0-21swapper/020:29:060
1341999644,13cyclictest0-21swapper/223:38:452
1341899644,53cyclictest0-21swapper/122:19:131
1341999634,51cyclictest0-21swapper/222:14:082
1341999634,51cyclictest0-21swapper/222:14:082
73242612,51sleep00-21swapper/023:04:090
243799610,4rtkit-daemon2436-21rtkit-daemon22:12:112
1341999615,15cyclictest0-21swapper/222:34:042
1341999614,15cyclictest0-21swapper/223:44:072
1341999614,13cyclictest0-21swapper/222:04:102
13418996144,12cyclictest0-21swapper/123:09:081
1341699613,52cyclictest0-21swapper/019:24:530
83412605,47sleep20-21swapper/221:59:102
243799600,4rtkit-daemon2436-21rtkit-daemon23:07:462
243799600,4rtkit-daemon2436-21rtkit-daemon21:56:042
243799600,4rtkit-daemon2436-21rtkit-daemon19:33:472
1341999604,16cyclictest0-21swapper/222:54:172
1341999604,16cyclictest0-21swapper/222:44:042
1341999604,15cyclictest0-21swapper/222:24:182
1341899603,50cyclictest16696-21runrttasks20:00:511
1341999595,15cyclictest0-21swapper/221:05:572
1341999594,16cyclictest0-21swapper/219:44:262
1341999594,15cyclictest0-21swapper/220:04:112
1341999594,15cyclictest0-21swapper/220:00:472
1341999594,12cyclictest0-21swapper/220:23:542
13419995922,14cyclictest0-21swapper/219:49:242
1341899594,48cyclictest0-21swapper/123:38:461
1341899594,48cyclictest0-21swapper/119:14:231
1341899594,47cyclictest4073-21taskset21:49:181
13416995932,11cyclictest0-21swapper/023:33:460
1341699593,50cyclictest0-21swapper/023:34:120
1341999585,31cyclictest0-21swapper/223:39:142
1341999585,31cyclictest0-21swapper/223:39:142
1341999584,15cyclictest0-21swapper/222:49:212
1341999584,15cyclictest0-21swapper/222:19:132
1341999584,15cyclictest0-21swapper/221:09:052
1341999584,14cyclictest0-21swapper/219:03:132
13419995821,14cyclictest0-21swapper/220:12:112
13419995821,13cyclictest0-21swapper/223:14:182
13419995819,13cyclictest0-21swapper/220:49:152
1341899584,47cyclictest0-21swapper/123:54:071
1341899584,47cyclictest0-21swapper/122:44:011
1341899584,47cyclictest0-21swapper/121:17:141
1341899583,16cyclictest0-21swapper/123:39:051
1341899583,16cyclictest0-21swapper/123:39:051
1341899583,14cyclictest0-21swapper/122:24:081
1341899583,14cyclictest0-21swapper/119:09:221
13416995841,11cyclictest0-21swapper/020:28:530
1341699584,48cyclictest0-21swapper/020:49:060
13416995832,12cyclictest0-21swapper/023:43:340
13416995832,12cyclictest0-21swapper/023:43:340
1341999577,13cyclictest0-21swapper/219:20:412
1341999575,29cyclictest0-21swapper/223:51:232
1341999575,13cyclictest0-21swapper/200:09:102
1341999574,15cyclictest0-21swapper/222:29:152
1341999574,15cyclictest0-21swapper/221:44:222
1341999574,15cyclictest0-21swapper/219:14:342
1341999574,14cyclictest0-21swapper/221:14:142
1341999574,13cyclictest0-21swapper/223:28:472
1341999574,13cyclictest0-21swapper/220:54:122
1341999574,13cyclictest0-21swapper/220:54:122
13419995719,14cyclictest0-21swapper/219:38:552
1341899575,13cyclictest0-21swapper/120:48:521
1341899574,11cyclictest1369-21runrttasks23:48:551
13418995739,12cyclictest0-21swapper/123:29:211
1341899573,12cyclictest0-21swapper/100:17:561
1341699575,46cyclictest0-21swapper/019:09:090
13416995737,12cyclictest0-21swapper/023:54:040
1341699573,8cyclictest27134-21grep23:44:200
1341999565,29cyclictest0-21swapper/221:36:032
1341999565,29cyclictest0-21swapper/221:36:032
1341999564,46cyclictest0-21swapper/223:19:112
1341999564,29cyclictest0-21swapper/220:29:112
1341999564,29cyclictest0-21swapper/220:14:272
1341999564,29cyclictest0-21swapper/220:14:272
1341999564,17cyclictest0-21swapper/222:59:002
1341999564,17cyclictest0-21swapper/222:59:002
1341999564,15cyclictest0-21swapper/219:24:122
1341899564,45cyclictest0-21swapper/119:01:191
1341899563,29cyclictest0-21swapper/118:54:231
13418995624,13cyclictest0-21swapper/100:04:131
1341699563,47cyclictest0-21swapper/022:19:140
243799550,5rtkit-daemon2436-21rtkit-daemon22:43:402
1341999555,13cyclictest0-21swapper/200:17:062
1341999554,44cyclictest0-21swapper/200:03:592
1341999554,29cyclictest0-21swapper/221:39:092
1341999554,27cyclictest0-21swapper/219:39:082
1341999554,14cyclictest0-21swapper/223:09:152
13419995517,13cyclictest0-21swapper/220:44:072
1341899553,28cyclictest0-21swapper/100:02:031
1341899553,26cyclictest26944-21ls20:24:091
1341899553,13cyclictest0-21swapper/119:53:551
1341899553,12cyclictest0-21swapper/123:24:011
13416995531,11cyclictest0-21swapper/019:39:230
1341699553,8cyclictest15422-21taskset23:21:180
1341699553,46cyclictest0-21swapper/023:48:550
1341699553,46cyclictest0-21swapper/021:56:050
1341699553,12cyclictest0-21swapper/020:14:190
1341699553,12cyclictest0-21swapper/020:14:190
1341699553,10cyclictest0-21swapper/023:23:580
1341999545,12cyclictest16081-21aten2.4-expect18:54:102
1341999544,28cyclictest0-21swapper/223:54:042
13419995435,10cyclictest12921-21grep20:59:272
13419995421,10cyclictest1056-21smartctl23:59:052
13419995414,14cyclictest0-21swapper/219:13:512
1341899543,29cyclictest0-21swapper/122:39:011
13416995440,8cyclictest22720-21grep22:29:080
1341699544,44cyclictest0-21swapper/020:38:530
1341699544,44cyclictest0-21swapper/019:54:080
13416995430,10cyclictest0-21swapper/018:54:330
1341699542,10cyclictest0-21swapper/018:59:220
274572533,43sleep10-21swapper/123:47:111
1341999535,26cyclictest0-21swapper/221:19:062
1341999534,26cyclictest0-21swapper/220:39:062
1341999534,25cyclictest0-21swapper/218:49:232
1341999534,25cyclictest0-21swapper/218:49:232
1341999534,12cyclictest2796-21aten2.4-expect21:49:012
1341899534,27cyclictest0-21swapper/120:34:041
1341899533,43cyclictest0-21swapper/120:10:491
1341699536,40cyclictest0-21swapper/000:12:380
1341699535,9cyclictest0-21swapper/019:14:150
1341699533,45cyclictest0-21swapper/019:34:140
1341699533,44cyclictest0-21swapper/021:39:030
1341699533,10cyclictest0-21swapper/021:49:140
1341999524,26cyclictest0-21swapper/221:24:042
1341999524,11cyclictest20818-21aten2.4-expect19:04:082
1341999523,26cyclictest0-21swapper/221:29:052
1341999522,44cyclictest0-21swapper/223:30:302
1341899525,29cyclictest0-21swapper/120:50:551
1341899523,43cyclictest0-21swapper/120:54:261
1341899523,43cyclictest0-21swapper/120:54:261
1341899523,43cyclictest0-21swapper/120:29:071
1341899523,42cyclictest0-21swapper/122:38:071
1341899523,25cyclictest0-21swapper/121:04:241
13418995222,12cyclictest0-21swapper/120:59:251
13418995221,13cyclictest0-21swapper/123:19:101
13418995220,14cyclictest0-21swapper/120:15:491
13418995220,14cyclictest0-21swapper/120:15:491
1341899522,44cyclictest0-21swapper/118:49:171
1341899522,44cyclictest0-21swapper/118:49:171
1341699524,42cyclictest0-21swapper/018:49:090
1341699524,42cyclictest0-21swapper/018:49:090
13416995237,10cyclictest0-21swapper/023:09:170
1341699523,43cyclictest0-21swapper/021:29:190
1341699523,11cyclictest0-21swapper/022:03:590
131982522,9sleep00-21swapper/023:18:320
96492512,10sleep00-21swapper/022:03:190
243799510,5rtkit-daemon2436-21rtkit-daemon00:00:293
1341999513,25cyclictest0-21swapper/220:24:072
1341899519,13cyclictest0-21swapper/121:19:141
1341899513,44cyclictest0-21swapper/121:39:261
1341899513,41cyclictest0-21swapper/122:58:581
1341899513,41cyclictest0-21swapper/122:58:581
13418995121,12cyclictest0-21swapper/121:26:001
1341899512,44cyclictest0-21swapper/122:09:121
1341899512,43cyclictest0-21swapper/120:39:071
1341699515,10cyclictest0-21swapper/022:46:140
1341699514,42cyclictest0-21swapper/020:05:480
13416995134,12cyclictest0-21swapper/019:49:210
13416995133,12cyclictest0-21swapper/020:59:060
1341699513,8cyclictest28818-21awk22:43:470
1341699513,42cyclictest0-21swapper/019:29:100
80372504,10sleep00-21swapper/000:14:020
243799500,4rtkit-daemon2436-21rtkit-daemon21:29:203
1341899505,28cyclictest0-21swapper/120:04:081
1341899504,9cyclictest0-21swapper/122:29:141
1341899504,39cyclictest0-21swapper/119:58:551
1341899503,42cyclictest0-21swapper/119:04:261
1341899503,41cyclictest0-21swapper/122:53:581
1341899503,41cyclictest0-21swapper/122:17:451
1341899503,41cyclictest0-21swapper/122:17:451
1341899503,40cyclictest0-21swapper/119:29:091
13418995024,13cyclictest0-21swapper/122:04:051
13418995022,13cyclictest0-21swapper/123:14:011
13418995022,12cyclictest0-21swapper/119:44:071
1341899502,42cyclictest0-21swapper/121:09:231
1341899502,42cyclictest0-21swapper/119:19:081
1341699504,29cyclictest0-21swapper/020:48:520
1341699504,10cyclictest0-21swapper/019:05:390
13416995035,9cyclictest28409-21df21:34:060
13416995035,9cyclictest28409-21df21:34:060
13416995034,11cyclictest15559-21gltestperf22:14:110
13416995034,11cyclictest15559-21gltestperf22:14:110
13416995032,11cyclictest0-21swapper/019:59:270
1341699503,11cyclictest0-21swapper/020:39:060
13416995029,10cyclictest0-21swapper/022:09:000
204412492,10sleep00-21swapper/021:18:290
13420994927,9cyclictest21943-21munin-node20:14:273
13420994927,9cyclictest21943-21munin-node20:14:273
13420994921,13cyclictest0-21swapper/322:03:493
1341899494,39cyclictest0-21swapper/123:08:481
1341899494,39cyclictest0-21swapper/121:59:131
1341899494,39cyclictest0-21swapper/121:29:121
1341899494,24cyclictest25053-21perl20:19:111
1341899493,41cyclictest0-21swapper/121:44:001
1341899493,41cyclictest0-21swapper/119:39:081
1341899493,40cyclictest0-21swapper/122:49:001
1341899493,40cyclictest0-21swapper/121:54:031
1341899493,40cyclictest0-21swapper/119:34:121
13416994934,10cyclictest646-21unixbench-2d22:49:210
1341699493,39cyclictest0-21swapper/022:38:480
13420994827,10cyclictest0-21swapper/322:39:193
1341899484,14cyclictest0-21swapper/100:08:571
1341899483,40cyclictest0-21swapper/121:34:031
1341899483,40cyclictest0-21swapper/121:34:031
1341699484,10cyclictest0-21swapper/021:04:060
1341699484,10cyclictest0-21swapper/000:03:590
1341699483,37cyclictest0-21swapper/020:19:090
1341699482,22cyclictest1243-21snmpd19:23:140
13416994812,20cyclictest114550irq/46-eth020:09:190
87692472,39sleep00-21swapper/020:53:590
87692472,39sleep00-21swapper/020:53:590
13420994721,11cyclictest0-21swapper/320:34:083
1341899474,36cyclictest0-21swapper/118:48:581
1341699473,21cyclictest0-21swapper/021:44:100
13420994624,7cyclictest20565-21munin-run19:03:573
13420994619,12cyclictest0-21swapper/320:53:523
1341699463,28cyclictest0-21swapper/022:54:170
1341699462,37cyclictest0-21swapper/022:24:060
1341699462,25cyclictest1369-21runrttasks21:09:150
13420994532,9cyclictest0-21swapper/320:47:533
13420994521,18cyclictest7475-21aten_r1power_cu00:13:563
1341699453,37cyclictest0-21swapper/021:19:090
131562444,34sleep20-21swapper/218:46:032
13420994223,9cyclictest0-21swapper/321:07:533
1341699413,32cyclictest0-21swapper/018:48:580
13420993612,9cyclictest0-21swapper/318:48:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional