You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 12:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Tue Jul 07, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15054992623,2cyclictest6490-21ssh23:30:512
15054992622,4cyclictest0-21swapper/222:51:142
1505999250,24cyclictest0-21swapper/321:52:093
1505499250,24cyclictest0-21swapper/223:27:362
1505499250,24cyclictest0-21swapper/221:18:512
1505499250,22cyclictest31944-21apt-config22:00:012
1505099250,1cyclictest6669-21ssh00:27:151
15049992524,1cyclictest62550irq/127-eno122:44:100
15049992523,1cyclictest0-21swapper/023:01:450
1504999250,24cyclictest6965-21diskmemload23:22:020
1504999250,24cyclictest17122-21ssh21:18:150
15059992423,1cyclictest331rcuc/322:56:373
1505999240,2cyclictest22072-21ssh23:16:333
1505999240,23cyclictest7223-40mandb00:00:013
1505999240,23cyclictest10333-21sh21:11:433
1505999240,0cyclictest151rcu_preempt23:47:333
1505999240,0cyclictest0-21swapper/323:34:093
15054992424,0cyclictest0-21swapper/200:25:412
15054992422,1cyclictest6965-21diskmemload22:06:182
1505499240,0cyclictest0-21swapper/200:12:302
15050992423,1cyclictest0-21swapper/122:02:531
15050992422,1cyclictest0-21swapper/100:16:431
15050992421,2cyclictest29670-21ssh23:21:541
1505099240,23cyclictest0-21swapper/121:55:461
1505099240,23cyclictest0-21swapper/119:56:331
1505099240,22cyclictest0-21swapper/123:44:141
15049992424,0cyclictest6965-21diskmemload23:57:090
15049992423,1cyclictest9994-21chrt00:01:380
15049992423,1cyclictest62550irq/127-eno121:21:050
1504999240,24cyclictest6965-21diskmemload23:18:310
1504999240,24cyclictest0-21swapper/021:40:570
1504999240,24cyclictest0-21swapper/000:34:420
1504999240,23cyclictest6965-21diskmemload22:57:170
1504999240,23cyclictest22195-21cat22:20:000
1504999240,1cyclictest0-21swapper/022:06:310
1504999240,0cyclictest6965-21diskmemload22:11:470
1504999240,0cyclictest6965-21diskmemload00:09:200
15059992323,0cyclictest0-21swapper/323:54:333
15059992323,0cyclictest0-21swapper/323:27:173
15059992323,0cyclictest0-21swapper/323:07:013
15059992323,0cyclictest0-21swapper/323:02:223
15059992323,0cyclictest0-21swapper/322:36:513
15059992323,0cyclictest0-21swapper/322:30:233
15059992323,0cyclictest0-21swapper/321:46:153
15059992323,0cyclictest0-21swapper/321:43:083
15059992323,0cyclictest0-21swapper/300:31:173
15059992323,0cyclictest0-21swapper/300:01:563
15059992322,1cyclictest34-21ksoftirqd/323:35:163
1505999232,21cyclictest34-21ksoftirqd/322:00:013
15059992321,1cyclictest0-21swapper/322:05:483
15059992321,1cyclictest0-21swapper/321:15:013
1505999230,23cyclictest0-21swapper/322:52:273
1505999230,23cyclictest0-21swapper/322:48:153
1505999230,23cyclictest0-21swapper/321:34:253
1505999230,22cyclictest6965-21diskmemload23:12:343
1505999230,22cyclictest6965-21diskmemload00:39:143
1505999230,22cyclictest25273-21awk19:35:013
1505999230,1cyclictest0-21swapper/300:15:553
1505999230,0cyclictest151rcu_preempt19:25:013
1505999230,0cyclictest0-21swapper/322:19:363
1505999230,0cyclictest0-21swapper/300:05:383
15054992323,0cyclictest6965-21diskmemload23:54:272
15054992323,0cyclictest0-21swapper/222:10:162
15054992323,0cyclictest0-21swapper/200:22:262
15054992321,1cyclictest3483-21apt-get21:35:022
15054992321,1cyclictest0-21swapper/222:26:352
15054992321,1cyclictest0-21swapper/222:20:282
15054992321,1cyclictest0-21swapper/221:13:152
15054992321,1cyclictest0-21swapper/200:06:032
1505499230,2cyclictest7216-21cat22:35:012
1505499230,23cyclictest6965-21diskmemload23:16:532
1505499230,23cyclictest0-21swapper/223:41:152
1505499230,23cyclictest0-21swapper/223:24:202
1505499230,22cyclictest9004-21ssh00:30:062
1505499230,22cyclictest8982-21ssh23:05:002
1505499230,22cyclictest783-21snmpd21:00:022
1505499230,22cyclictest7199-21perf00:00:002
1505499230,22cyclictest379-21rm22:00:592
1505499230,22cyclictest2859-21munin-run21:35:002
1505499230,22cyclictest26949-21apt20:45:002
1505499230,22cyclictest25886-21turbostat.cron21:55:012
1505499230,22cyclictest22982-21ssh23:45:062
1505499230,22cyclictest19933-21ssh23:14:332
1505499230,22cyclictest18839-21ssh22:45:132
1505499230,22cyclictest16880-21ssh22:42:582
1505499230,22cyclictest0-21swapper/223:35:242
1505499230,22cyclictest0-21swapper/222:18:342
1505499230,22cyclictest0-21swapper/221:46:432
1505499230,22cyclictest0-21swapper/221:27:132
1505499230,22cyclictest0-21swapper/200:01:262
1505499230,0cyclictest0-21swapper/200:38:182
15050992323,0cyclictest0-21swapper/123:57:231
15050992323,0cyclictest0-21swapper/123:38:211
15050992323,0cyclictest0-21swapper/123:26:131
15050992323,0cyclictest0-21swapper/122:53:071
15050992323,0cyclictest0-21swapper/121:50:131
15050992323,0cyclictest0-21swapper/121:20:421
15050992321,1cyclictest6965-21diskmemload23:01:361
15050992321,1cyclictest28083-21ssh23:49:451
15050992321,1cyclictest0-21swapper/122:35:321
15050992321,1cyclictest0-21swapper/100:30:241
15050992320,2cyclictest14159-21ssh22:11:521
1505099231,21cyclictest6965-21diskmemload23:50:461
1505099230,23cyclictest22-21ksoftirqd/121:10:461
1505099230,23cyclictest0-21swapper/122:56:091
1505099230,23cyclictest0-21swapper/122:34:291
1505099230,22cyclictest7589-21ssh00:00:061
1505099230,22cyclictest6965-21diskmemload22:43:331
1505099230,22cyclictest30739-21ssh00:20:551
1505099230,22cyclictest19386-21rm21:48:521
1505099230,22cyclictest17216-21ssh21:18:231
1505099230,22cyclictest16969-21sh23:11:361
1505099230,22cyclictest0-21swapper/122:25:181
1505099230,1cyclictest0-21swapper/122:16:161
1505099230,0cyclictest0-21swapper/122:45:341
1505099230,0cyclictest0-21swapper/121:41:521
15049992323,0cyclictest0-21swapper/023:43:470
15049992323,0cyclictest0-21swapper/000:39:140
15049992322,1cyclictest6965-21diskmemload22:32:090
15049992322,1cyclictest62550irq/127-eno122:50:210
15049992322,1cyclictest29234-21ssh23:51:020
15049992322,1cyclictest27155-21ssh21:56:280
15049992321,1cyclictest23258-21apt-get19:31:310
15049992321,1cyclictest0-21swapper/023:05:500
15049992321,1cyclictest0-21swapper/022:21:410
15049992320,2cyclictest24864-21sh23:46:350
15049992320,2cyclictest15690-21perf19:15:000
15049992318,4cyclictest1018-21perf22:30:000
1504999230,2cyclictest3866-21ssh21:35:260
1504999230,23cyclictest6965-21diskmemload00:11:280
1504999230,23cyclictest0-21swapper/022:35:540
1504999230,23cyclictest0-21swapper/021:10:310
1504999230,23cyclictest0-21swapper/000:20:050
1504999230,22cyclictest9937-21ssh23:33:150
1504999230,22cyclictest3914-21sh23:27:430
1504999230,22cyclictest25201-21ssh00:16:030
1504999230,22cyclictest24486-21rm21:25:180
1504999230,22cyclictest0-21swapper/023:35:200
1504999230,22cyclictest0-21swapper/022:45:300
1504999230,21cyclictest4213-21ssh00:26:220
1504999230,0cyclictest0-21swapper/021:47:130
15059992221,1cyclictest0-21swapper/322:40:383
15059992220,1cyclictest0-21swapper/322:10:553
15059992220,1cyclictest0-21swapper/300:26:003
1505999220,22cyclictest6965-21diskmemload21:20:593
1505999220,22cyclictest0-21swapper/323:44:573
1505999220,21cyclictest6965-21diskmemload21:35:583
1505999220,21cyclictest0-21swapper/321:25:473
1505999220,21cyclictest0-21swapper/300:21:323
1505999220,1cyclictest0-21swapper/322:00:023
1505999220,0cyclictest0-21swapper/322:27:553
1505999220,0cyclictest0-21swapper/300:11:133
15054992222,0cyclictest6965-21diskmemload22:31:222
15054992221,1cyclictest0-21swapper/221:20:432
15054992220,1cyclictest30271-21sh22:55:352
15054992220,1cyclictest18091-21apt19:20:002
1505499220,21cyclictest0-21swapper/223:05:362
1505499220,21cyclictest0-21swapper/220:21:332
1505499220,21cyclictest0-21swapper/220:16:312
1505499220,1cyclictest24485-21ssh00:15:092
1505499220,0cyclictest6965-21diskmemload21:41:462
15050992222,0cyclictest6965-21diskmemload00:36:121
15050992222,0cyclictest0-21swapper/121:30:481
15050992221,1cyclictest6965-21diskmemload23:30:091
15050992220,1cyclictest19068-21ssh00:10:311
15050992220,1cyclictest0-21swapper/123:15:051
1505099220,22cyclictest6965-21diskmemload23:06:471
1505099220,21cyclictest6965-21diskmemload22:05:351
1505099220,21cyclictest6965-21diskmemload21:35:331
1505099220,21cyclictest0-21swapper/121:25:101
1505099220,21cyclictest0-21swapper/119:25:011
1505099220,21cyclictest0-21swapper/100:05:221
1505099220,1cyclictest20864-21apt-get19:26:321
1505099220,1cyclictest0-21swapper/122:22:051
15049992220,1cyclictest32460-21chrt22:00:220
1504999220,22cyclictest6965-21diskmemload23:10:400
1504999220,21cyclictest0-21swapper/020:06:310
1504999220,1cyclictest2708-21ssh21:34:520
1504999220,1cyclictest23886-21sh21:52:150
15059992121,0cyclictest20114-21apt-get20:31:313
15059992120,1cyclictest2513-21apt-get21:01:333
15059992120,1cyclictest16047-21apt-get19:16:303
15059992120,0cyclictest9726-21apt-get20:10:023
1505999210,20cyclictest0-21swapper/320:36:323
1505999210,20cyclictest0-21swapper/320:26:333
1505999210,1cyclictest5319-21apt-get20:01:323
1505999210,1cyclictest24928-21apt-get20:41:323
15054992120,0cyclictest0-21swapper/220:26:332
1505499210,20cyclictest32111-21apt-get20:56:342
1505499210,20cyclictest0-21swapper/219:51:322
15050992121,0cyclictest0-21swapper/120:01:321
15050992120,1cyclictest22512-21apt-get20:36:321
1505099210,20cyclictest17722-21apt-get20:26:331
1505099210,20cyclictest0-21swapper/121:06:331
1505099210,20cyclictest0-21swapper/120:41:321
1505099210,20cyclictest0-21swapper/119:16:301
1505099210,20cyclictest0-21swapper/119:15:001
15049992119,1cyclictest0-21swapper/020:56:340
1504999210,20cyclictest27322-21apt-get20:46:330
15059992020,0cyclictest0-21swapper/320:16:313
15059992018,1cyclictest29716-21apt-get20:51:333
1505999200,0cyclictest0-21swapper/319:51:313
1505999200,0cyclictest0-21swapper/319:26:333
15054992020,0cyclictest2848-21apt-get19:56:332
15054992020,0cyclictest0-21swapper/220:51:332
15054992020,0cyclictest0-21swapper/220:36:322
1505499200,20cyclictest0-21swapper/220:11:312
15050992020,0cyclictest0-21swapper/121:01:331
15050992017,2cyclictest1-21systemd20:20:011
15049992020,0cyclictest0-21swapper/021:01:320
15049992020,0cyclictest0-21swapper/019:26:320
15049992018,1cyclictest14950-21awk20:20:010
1504999200,0cyclictest0-21swapper/020:11:310
1504999200,0cyclictest0-21swapper/019:16:300
15059991919,0cyclictest0-21swapper/320:21:333
15054991919,0cyclictest0-21swapper/220:46:332
15054991919,0cyclictest0-21swapper/219:46:302
15054991919,0cyclictest0-21swapper/219:26:332
15054991918,1cyclictest0-21swapper/221:06:322
15050991919,0cyclictest0-21swapper/120:31:311
15049991918,1cyclictest30463-21apt-get19:46:300
15049991917,1cyclictest6965-21diskmemload21:10:010
15059991818,0cyclictest0-21swapper/320:06:313
15054991816,1cyclictest30086-21idleruntime-cro19:45:002
15050991818,0cyclictest0-21swapper/120:56:331
15050991818,0cyclictest0-21swapper/119:31:321
15050991816,1cyclictest2480-21perf19:55:001
15049991818,0cyclictest0-21swapper/019:21:330
15059991717,0cyclictest0-21swapper/320:46:333
15059991717,0cyclictest0-21swapper/319:56:333
15059991717,0cyclictest0-21swapper/319:36:323
15059991715,1cyclictest4960-21apt-get21:06:333
15054991717,0cyclictest0-21swapper/220:31:322
15054991716,1cyclictest25654-21apt-get19:36:312
15054991715,1cyclictest7713-21apt-get20:06:312
15054991711,5cyclictest611-21irqbalance19:33:252
15050991717,0cyclictest15330-21apt-get20:21:331
1505099170,16cyclictest25887-21munin-node20:46:211
15049991717,0cyclictest0-21swapper/020:41:320
15049991717,0cyclictest0-21swapper/020:00:000
15049991716,1cyclictest774-21runrttasks20:31:310
15059991616,0cyclictest0-21swapper/320:56:333
15054991616,0cyclictest0-21swapper/220:01:322
15050991614,1cyclictest31465-21needreboot19:46:341
1505099160,14cyclictest0-21swapper/119:36:581
1504999165,9cyclictest783-21snmpd19:35:350
15049991616,0cyclictest0-21swapper/020:51:330
15049991615,1cyclictest0-21swapper/020:21:330
15049991614,1cyclictest0-21swapper/020:05:010
15049991514,0cyclictest0-21swapper/019:51:310
15049991512,2cyclictest24524-21cron20:40:000
1505999140,14cyclictest0-21swapper/319:40:073
1505999140,14cyclictest0-21swapper/319:10:313
1505999140,0cyclictest0-21swapper/319:45:353
1505499140,14cyclictest0-21swapper/219:10:322
1505499140,0cyclictest0-21swapper/219:20:062
15050991414,0cyclictest0-21swapper/120:50:381
1505099140,14cyclictest0-21swapper/120:10:121
1505099140,14cyclictest0-21swapper/119:40:031
15049991414,0cyclictest0-21swapper/020:27:100
1504999140,14cyclictest0-21swapper/019:40:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional