You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 12:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Tue Dec 02, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24503999637,11cyclictest0-21swapper/120:25:071
24505999535,12cyclictest0-21swapper/221:35:222
24500999523,8cyclictest7663-21crond20:29:490
57012940,2sleep20-21swapper/221:30:172
2450099913,29cyclictest0-21swapper/021:25:210
2450099913,27cyclictest20154-21chrt19:45:270
193492910,3sleep3361ktimersoftd/319:45:143
24500999022,61cyclictest28996-21kworker/u8:100:05:070
24500999017,13cyclictest0-21swapper/023:39:420
2450099895,76cyclictest0-21swapper/022:09:530
247392880,6sleep3361ktimersoftd/321:05:033
247392880,6sleep3361ktimersoftd/321:05:033
24500998822,12cyclictest0-21swapper/023:15:100
24500998822,12cyclictest0-21swapper/023:15:100
24500998728,51cyclictest24501-21idleruntime-cro18:49:530
24500998629,9cyclictest30644-21munin-node20:10:030
24500998521,7cyclictest18478-21awk19:44:510
24500998521,7cyclictest18478-21awk19:44:510
24500998518,10cyclictest0-21swapper/022:14:590
2450099843,32cyclictest0-21swapper/020:05:150
24500998323,9cyclictest12673-21munin-node20:40:070
24500998228,10cyclictest19470-21wc00:15:020
2450099814,14cyclictest0-21swapper/023:49:410
2450099814,12cyclictest0-21swapper/019:16:380
2450099813,31cyclictest0-21swapper/022:25:070
24500998126,48cyclictest0-21swapper/022:55:070
24500998123,8cyclictest1369-21runrttasks20:15:520
2450099803,28cyclictest15315-21sshd20:45:030
24500998023,7cyclictest5192-21crond22:39:440
24500998019,12cyclictest0-21swapper/021:15:110
2450099794,28cyclictest0-21swapper/021:59:580
2450099793,29cyclictest9137-21munin-node19:25:120
24500997921,8cyclictest1243-21snmpd23:22:210
24500997919,9cyclictest0-21swapper/022:30:070
24500997919,9cyclictest0-21swapper/022:30:070
24500997919,15cyclictest0-21swapper/021:34:470
24500997915,17cyclictest0-21swapper/019:50:260
2450099785,29cyclictest0-21swapper/023:34:420
2450099784,67cyclictest0-21swapper/023:04:560
2450099784,31cyclictest0-21swapper/019:35:160
2450099784,29cyclictest0-21swapper/022:45:100
2450099783,29cyclictest13538-21latency_hist00:04:420
2450099783,29cyclictest13538-21latency_hist00:04:420
24500997822,8cyclictest14230-21cstates21:50:000
24500997818,12cyclictest0-21swapper/019:30:160
24500997812,16cyclictest0-21swapper/020:35:070
2450099779,18cyclictest0-21swapper/019:55:060
2450099774,14cyclictest0-21swapper/020:20:030
2450099773,28cyclictest28198-21ntpdc23:25:090
2450099773,15cyclictest0-21swapper/021:10:080
24500997725,8cyclictest1369-21runrttasks21:35:010
24500997719,7cyclictest15176-21kworker/0:120:59:590
2450099764,14cyclictest0-21swapper/020:30:230
2450099763,32cyclictest1243-21snmpd22:07:070
2450099763,28cyclictest1243-21snmpd19:11:370
2450099763,27cyclictest19926-21taskset23:09:580
2450099763,14cyclictest0-21swapper/021:20:020
24500997625,8cyclictest25225-21entropy21:05:080
24500997625,8cyclictest25225-21entropy21:05:080
24500997624,8cyclictest21853-21sshd20:57:540
2450099762,31cyclictest18086-21sshd21:55:590
24500997616,7cyclictest11261-21latency_hist23:59:420
24500997616,7cyclictest11261-21latency_hist23:59:420
2450099754,31cyclictest0-21swapper/023:50:050
2450099753,8cyclictest10932-21sshd21:41:000
2450099753,66cyclictest0-21swapper/021:44:590
2450099753,22cyclictest0-21swapper/018:51:040
24500997524,8cyclictest28236-21missed_timers18:55:180
24500997524,8cyclictest28236-21missed_timers18:55:180
24500997523,10cyclictest6758-21munin-node19:20:270
24500997522,8cyclictest11354-21missed_timers22:50:080
24500997512,13cyclictest0-21swapper/022:40:140
2450099743,13cyclictest0-21swapper/020:54:490
24500997416,12cyclictest0-21swapper/020:00:080
2450099735,31cyclictest0-21swapper/019:00:090
24500997322,8cyclictest1858-21cpu23:39:550
24500997322,11cyclictest1629-21nscd19:09:530
24500997318,10cyclictest0-21swapper/022:20:090
2450099723,15cyclictest0-21swapper/000:13:330
24500997124,8cyclictest14994-21munin-node23:00:040
24503996739,13cyclictest0-21swapper/123:19:561
24503996538,12cyclictest0-21swapper/122:54:561
8902643,53sleep20-21swapper/219:05:272
24503996439,11cyclictest0-21swapper/121:39:471
24503996438,11cyclictest0-21swapper/123:04:561
2450799634,51cyclictest0-21swapper/320:45:213
24505996335,12cyclictest0-21swapper/222:59:442
2450599633,54cyclictest4058-21munin-run21:29:472
2450599632,55cyclictest32642-21sshd22:28:582
24503996336,11cyclictest14994-21munin-node22:59:591
2450599623,50cyclictest25002-21cpuspeed_turbos21:05:052
2450599623,50cyclictest25002-21cpuspeed_turbos21:05:052
2450599622,54cyclictest0-21swapper/223:30:182
2450399624,52cyclictest0-21swapper/123:29:431
2450799614,50cyclictest0-21swapper/319:35:083
2450799614,49cyclictest0-21swapper/300:02:163
2450799614,49cyclictest0-21swapper/300:02:163
24507996114,39cyclictest2799-21kworker/3:223:39:543
24503996133,11cyclictest0-21swapper/121:33:151
2450799604,49cyclictest0-21swapper/323:14:563
2450799604,49cyclictest0-21swapper/323:14:563
24507996023,31cyclictest11172-21gltestperf20:35:123
2450599608,47cyclictest0-21swapper/222:00:192
2450599603,51cyclictest0-21swapper/219:20:052
24503996036,9cyclictest0-21swapper/122:10:211
2450399603,51cyclictest0-21swapper/120:35:081
2450799595,47cyclictest8036-21sendmail_mailqu23:50:133
2450599593,49cyclictest26872-21latency_hist18:54:542
2450599593,47cyclictest32084-21cpuspeed_turbos23:34:582
24503995936,9cyclictest9137-21munin-node19:25:251
24503995933,11cyclictest0-21swapper/120:45:071
117372595,12sleep20-21swapper/220:35:242
2450799584,48cyclictest0-21swapper/320:15:023
2450799584,47cyclictest27461-21df_abs21:10:033
2450799584,47cyclictest0-21swapper/321:59:473
2450799583,49cyclictest0-21swapper/323:05:083
24507995820,11cyclictest0-21swapper/320:21:483
24505995831,12cyclictest0-21swapper/219:30:062
2450599583,49cyclictest0-21swapper/223:40:032
2450599582,50cyclictest0-21swapper/222:19:572
24503995831,13cyclictest0-21swapper/122:25:041
2450399583,49cyclictest0-21swapper/100:12:281
24503995829,12cyclictest0-21swapper/123:59:421
24503995829,12cyclictest0-21swapper/123:59:421
108032583,48sleep110853-21fschecks_count22:50:021
2450799575,28cyclictest0-21swapper/322:07:073
2450799574,46cyclictest14029-21sendmail_mailqu22:55:143
2450799573,25cyclictest9056-21munin-node21:40:143
24507995719,11cyclictest0-21swapper/323:19:593
24507995717,11cyclictest0-21swapper/323:27:223
24505995730,11cyclictest0-21swapper/200:15:022
2450599573,49cyclictest0-21swapper/220:40:202
24503995734,9cyclictest30075-21diskstats19:00:101
2450399572,49cyclictest0-21swapper/100:06:391
2450799564,47cyclictest0-21swapper/319:30:263
2450799564,28cyclictest0-21swapper/322:20:123
2450799564,28cyclictest0-21swapper/320:00:083
24507995620,11cyclictest0-21swapper/319:06:363
24507995619,12cyclictest0-21swapper/321:30:073
2450599565,46cyclictest0-21swapper/220:20:012
2450599563,47cyclictest0-21swapper/219:01:062
2450399564,47cyclictest0-21swapper/119:30:041
24503995631,12cyclictest0-21swapper/122:03:301
24503995631,12cyclictest0-21swapper/121:53:141
24503995631,12cyclictest0-21swapper/119:10:141
24503995615,24cyclictest25815-21telnet22:14:591
243799560,4rtkit-daemon2436-21rtkit-daemon20:17:151
2450799555,28cyclictest0-21swapper/322:42:143
2450799555,27cyclictest0-21swapper/322:47:143
2450799555,27cyclictest0-21swapper/300:07:283
2450799553,24cyclictest31135-21diskstats22:25:023
24507995510,39cyclictest0-21swapper/321:59:583
2450599553,47cyclictest0-21swapper/221:20:172
2450599553,47cyclictest0-21swapper/219:45:102
2450399556,27cyclictest0-21swapper/119:06:361
2450399554,36cyclictest0-21swapper/119:15:421
2450399554,30cyclictest0-21swapper/120:40:191
24503995530,12cyclictest0-21swapper/122:34:151
24503995530,12cyclictest0-21swapper/122:34:151
24503995525,13cyclictest0-21swapper/123:35:111
311382544,43sleep10-21swapper/123:30:281
2450799545,41cyclictest0-21swapper/320:40:033
2450799544,44cyclictest0-21swapper/321:35:013
2450799544,44cyclictest0-21swapper/300:18:103
2450799544,28cyclictest0-21swapper/321:16:583
2450799543,11cyclictest0-21swapper/319:15:053
2450599545,44cyclictest0-21swapper/222:08:592
2450599545,44cyclictest0-21swapper/221:16:432
2450599545,44cyclictest0-21swapper/220:45:102
2450599543,46cyclictest0-21swapper/222:40:172
2450599543,44cyclictest12885-21fgrep21:45:162
2450599543,44cyclictest0-21swapper/223:10:142
2450599542,47cyclictest0-21swapper/219:15:202
2450399545,31cyclictest0-21swapper/120:00:281
2450399544,29cyclictest0-21swapper/100:00:131
2450399544,29cyclictest0-21swapper/100:00:131
24503995429,12cyclictest0-21swapper/119:44:471
24503995429,12cyclictest0-21swapper/119:44:471
24503995423,14cyclictest0-21swapper/123:49:571
24503995422,14cyclictest0-21swapper/121:55:181
24503995422,13cyclictest0-21swapper/120:30:081
24503995422,13cyclictest0-21swapper/118:50:201
24503995420,14cyclictest0-21swapper/100:14:581
2450799534,43cyclictest0-21swapper/323:30:033
2450799534,43cyclictest0-21swapper/319:55:033
24507995338,9cyclictest23995-21cut22:10:043
2450799533,28cyclictest0-21swapper/321:20:163
2450599533,45cyclictest0-21swapper/219:25:052
2450599533,44cyclictest0-21swapper/220:15:012
2450599533,42cyclictest0-21swapper/222:34:442
2450599533,42cyclictest0-21swapper/222:34:442
2450599532,46cyclictest0-21swapper/220:06:342
24505995318,28cyclictest1369-21runrttasks22:59:552
2450399536,29cyclictest0-21swapper/122:35:091
2450399535,29cyclictest0-21swapper/123:40:081
2450399534,32cyclictest0-21swapper/118:55:131
2450399534,32cyclictest0-21swapper/118:55:131
2450399534,31cyclictest0-21swapper/123:15:051
2450399534,31cyclictest0-21swapper/123:15:051
2450399534,30cyclictest0-21swapper/123:45:141
2450399534,30cyclictest0-21swapper/122:44:581
2450399534,30cyclictest0-21swapper/121:10:041
2450399533,25cyclictest0-21swapper/120:10:221
24503995324,13cyclictest0-21swapper/119:55:191
24503995320,26cyclictest19922-21tail23:09:581
133902533,44sleep10-21swapper/121:46:581
265742526,10sleep20-21swapper/223:23:122
2450799527,39cyclictest0-21swapper/300:12:283
2450799525,41cyclictest0-21swapper/320:25:053
2450799524,42cyclictest0-21swapper/319:44:513
2450799524,42cyclictest0-21swapper/319:44:513
2450799524,30cyclictest0-21swapper/320:05:053
2450599523,44cyclictest0-21swapper/219:57:142
24505995227,10cyclictest0-21swapper/219:14:522
2450599522,44cyclictest0-21swapper/223:05:142
2450399525,29cyclictest0-21swapper/121:25:141
2450399525,29cyclictest0-21swapper/121:08:021
2450399525,29cyclictest0-21swapper/121:08:021
2450399525,29cyclictest0-21swapper/119:50:091
2450399525,28cyclictest0-21swapper/122:22:091
2450399525,27cyclictest0-21swapper/119:46:431
2450399524,42cyclictest0-21swapper/122:06:541
24503995217,16cyclictest0-21swapper/119:20:131
168782522,10sleep30-21swapper/323:01:203
2450799514,41cyclictest361ktimersoftd/322:30:023
2450799514,41cyclictest361ktimersoftd/322:30:023
2450799514,16cyclictest0-21swapper/318:55:163
2450799514,16cyclictest0-21swapper/318:55:163
24507995136,9cyclictest8260-21tr20:30:053
24507995135,10cyclictest14192-21cpuspeed_turbos21:50:003
2450799513,44cyclictest361ktimersoftd/320:10:223
2450799513,43cyclictest0-21swapper/321:25:103
24507995117,11cyclictest0-21swapper/323:47:243
24507995114,13cyclictest1369-21runrttasks19:20:233
2450599514,9cyclictest0-21swapper/221:55:092
24505995136,11cyclictest0-21swapper/220:10:442
2450599513,42cyclictest0-21swapper/223:59:572
2450599513,42cyclictest0-21swapper/223:59:572
2450599513,42cyclictest0-21swapper/200:10:122
2450599513,41cyclictest9217-21cpu23:54:552
2450599513,41cyclictest9217-21cpu23:54:552
24505995129,11cyclictest0-21swapper/220:27:342
2450599512,43cyclictest0-21swapper/222:49:552
2450399515,28cyclictest0-21swapper/122:39:581
243799510,5rtkit-daemon2436-21rtkit-daemon19:36:541
2450799507,27cyclictest0-21swapper/323:12:193
2450799504,31cyclictest0-21swapper/319:25:253
2450799504,26cyclictest0-21swapper/322:50:133
2450799504,26cyclictest0-21swapper/319:00:213
2450799504,26cyclictest0-21swapper/318:50:293
2450599504,12cyclictest0-21swapper/219:35:082
2450599503,41cyclictest0-21swapper/223:44:582
2450599503,41cyclictest0-21swapper/222:35:032
2450599503,41cyclictest0-21swapper/222:15:002
2450599503,41cyclictest0-21swapper/220:04:512
2450599502,44cyclictest0-21swapper/219:40:192
2450599502,44cyclictest0-21swapper/219:40:192
2450599502,43cyclictest0-21swapper/220:55:152
2450599502,43cyclictest0-21swapper/220:34:492
2450599502,43cyclictest0-21swapper/200:05:002
2450599502,42cyclictest24013-21sshd23:16:002
2450599502,42cyclictest24013-21sshd23:16:002
2450399505,26cyclictest19856-21munin-node20:55:021
2450399504,28cyclictest0-21swapper/121:40:031
2450399504,28cyclictest0-21swapper/120:20:171
24503995032,10cyclictest1136-21unixbench_singl21:20:241
24503995016,12cyclictest0-21swapper/121:19:491
240982503,11sleep10-21swapper/121:00:591
196162503,10sleep10-21swapper/120:54:461
11002502,10sleep30-21swapper/323:35:263
99132493,40sleep20-21swapper/222:48:502
2450799495,15cyclictest0-21swapper/321:01:563
2450599495,39cyclictest0-21swapper/223:51:372
2450599495,38cyclictest0-21swapper/221:09:592
24505994935,8cyclictest22860-21tune2fs21:00:062
2450599493,41cyclictest25553-21latency_hist22:14:452
2450599493,40cyclictest0-21swapper/221:49:572
40112482,10sleep30-21swapper/319:13:223
2450799485,38cyclictest0-21swapper/320:55:023
2450799484,14cyclictest0-21swapper/320:54:493
2450599483,39cyclictest0-21swapper/218:55:212
2450599483,39cyclictest0-21swapper/218:55:212
2450599483,38cyclictest0-21swapper/221:42:562
24505994818,11cyclictest0-21swapper/223:24:542
2450399484,29cyclictest0-21swapper/120:05:121
238462484,37sleep224017-21unixbench_multi18:45:212
230202482,9sleep30-21swapper/319:52:083
224332482,9sleep20-21swapper/219:50:222
2450799474,37cyclictest0-21swapper/318:49:543
2450799473,28cyclictest0-21swapper/321:45:123
24507994628,10cyclictest4794-21sed22:35:193
2450799462,10cyclictest0-21swapper/322:17:283
2450599464,27cyclictest0-21swapper/220:50:162
89982452,36sleep30-21swapper/323:54:533
89982452,36sleep30-21swapper/323:54:533
2450399364,25cyclictest0-21swapper/118:49:541
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional