You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 13:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Jun 28, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
596999260,2cyclictest18043-21expr23:20:482
596999260,2cyclictest18043-21expr23:20:482
597499250,25cyclictest0-21swapper/322:34:423
597499250,24cyclictest32021-21diskmemload22:25:563
5969992520,4cyclictest0-21swapper/222:13:262
596599250,24cyclictest16951-21ssh21:54:171
5961992524,0cyclictest0-21swapper/000:38:430
596199250,1cyclictest0-21swapper/022:59:020
5974992423,1cyclictest2118-21ssh22:38:203
597499240,24cyclictest32021-21diskmemload00:29:543
597499240,23cyclictest5910-21ssh23:38:433
597499240,23cyclictest4336-21ssh23:08:293
597499240,23cyclictest32021-21diskmemload00:05:413
597499240,23cyclictest19018-21awk22:25:003
597499240,23cyclictest0-21swapper/321:30:083
597499240,1cyclictest31394-21/usr/sbin/munin00:00:563
5969992420,3cyclictest19016-21turbostat.cron22:25:002
596999240,24cyclictest32021-21diskmemload22:15:262
596999240,23cyclictest14149-21apt-get00:15:502
596999240,23cyclictest0-21swapper/223:50:532
596999240,1cyclictest0-21swapper/222:54:262
596999240,0cyclictest32021-21diskmemload23:01:052
5965992424,0cyclictest0-21swapper/100:11:211
596599240,23cyclictest0-21swapper/122:32:201
596599240,0cyclictest32021-21diskmemload22:25:571
5961992424,0cyclictest62550irq/127-eno122:34:020
5961992422,1cyclictest0-21swapper/023:13:040
5961992421,2cyclictest13-21ksoftirqd/000:00:000
596199241,22cyclictest32021-21diskmemload00:25:500
596199240,24cyclictest0-21swapper/023:02:250
596199240,23cyclictest32021-21diskmemload23:46:210
596199240,23cyclictest32021-21diskmemload22:10:170
596199240,23cyclictest32021-21diskmemload21:39:270
596199240,23cyclictest0-21swapper/021:32:120
596199240,23cyclictest0-21swapper/000:16:000
596199240,1cyclictest0-21swapper/023:21:480
596199240,1cyclictest0-21swapper/023:21:480
596199240,1cyclictest0-21swapper/022:35:540
596199240,0cyclictest0-21swapper/023:27:120
5974992323,0cyclictest32021-21diskmemload21:16:043
5974992323,0cyclictest0-21swapper/323:13:333
5974992323,0cyclictest0-21swapper/322:53:223
5974992323,0cyclictest0-21swapper/321:41:473
5974992323,0cyclictest0-21swapper/321:35:083
5974992322,1cyclictest3076-21ssh00:34:223
5974992321,1cyclictest14283-21ssh21:22:093
5974992321,1cyclictest12787-21ssh22:46:303
5974992321,1cyclictest0-21swapper/323:45:133
5974992321,1cyclictest0-21swapper/321:57:043
5974992320,2cyclictest28381-21sh22:04:373
597499230,2cyclictest15672-21cat00:15:593
597499230,23cyclictest0-21swapper/323:02:223
597499230,23cyclictest0-21swapper/322:41:153
597499230,23cyclictest0-21swapper/322:19:083
597499230,23cyclictest0-21swapper/322:08:283
597499230,23cyclictest0-21swapper/300:20:493
597499230,22cyclictest32021-21diskmemload22:10:193
597499230,0cyclictest32021-21diskmemload21:52:223
597499230,0cyclictest15327-21ssh23:17:473
597499230,0cyclictest0-21swapper/323:43:493
597499230,0cyclictest0-21swapper/323:26:463
5969992323,0cyclictest0-21swapper/200:37:212
5969992320,2cyclictest31251-21rm00:30:502
5969992320,2cyclictest2182-21ssh00:04:592
5969992320,0cyclictest0-21swapper/223:32:182
5969992319,0cyclictest0-21swapper/221:45:172
596999230,2cyclictest29297-21ssh22:33:492
596999230,2cyclictest20769-21ssh21:28:162
596999230,23cyclictest0-21swapper/222:57:302
596999230,23cyclictest0-21swapper/222:43:432
596999230,23cyclictest0-21swapper/222:04:582
596999230,23cyclictest0-21swapper/221:36:062
596999230,23cyclictest0-21swapper/221:19:472
596999230,22cyclictest3581-21rm23:35:572
596999230,22cyclictest2739-21rm21:11:452
596999230,22cyclictest14918-21unixbench-2d23:45:592
596999230,22cyclictest13738-21sh22:47:542
596999230,22cyclictest0-21swapper/223:26:252
596999230,0cyclictest32021-21diskmemload21:56:212
596999230,0cyclictest0-21swapper/223:06:312
596999230,0cyclictest0-21swapper/200:21:592
5965992323,0cyclictest32021-21diskmemload22:21:171
5965992323,0cyclictest32021-21diskmemload21:15:081
5965992323,0cyclictest0-21swapper/123:34:131
5965992323,0cyclictest0-21swapper/122:52:361
5965992323,0cyclictest0-21swapper/121:31:401
5965992323,0cyclictest0-21swapper/121:13:521
5965992321,1cyclictest19366-21sh21:55:531
5965992321,1cyclictest15107-21apt-get20:35:471
5965992321,1cyclictest10746-21munin-run19:20:001
5965992321,1cyclictest0-21swapper/123:00:521
5965992320,2cyclictest7109-21ssh00:37:151
5965992320,2cyclictest21135-21tr00:20:581
5965992320,2cyclictest20198-21sh23:21:421
5965992320,2cyclictest20198-21sh23:21:421
596599230,23cyclictest32392-21ssh00:01:531
596599230,23cyclictest32021-21diskmemload21:27:311
596599230,23cyclictest0-21swapper/121:37:141
596599230,23cyclictest0-21swapper/121:21:371
596599230,22cyclictest3360-21users21:40:581
596599230,22cyclictest0-21swapper/122:45:081
596599230,22cyclictest0-21swapper/122:05:481
596599230,22cyclictest0-21swapper/100:25:051
596599230,1cyclictest32021-21diskmemload23:25:561
596599230,1cyclictest15236-21kworker/u8:222:04:071
596599230,1cyclictest15121-21sh23:17:311
596599230,0cyclictest32021-21diskmemload00:05:071
596599230,0cyclictest0-21swapper/123:44:141
596599230,0cyclictest0-21swapper/123:06:221
596599230,0cyclictest0-21swapper/122:39:281
5961992323,0cyclictest32021-21diskmemload00:22:240
5961992323,0cyclictest0-21swapper/023:51:560
5961992323,0cyclictest0-21swapper/023:36:340
5961992323,0cyclictest0-21swapper/022:25:490
5961992323,0cyclictest0-21swapper/022:18:370
5961992323,0cyclictest0-21swapper/021:57:010
5961992323,0cyclictest0-21swapper/021:52:190
5961992321,1cyclictest10523-21apt-get22:45:490
5961992321,1cyclictest0-21swapper/023:15:070
5961992321,1cyclictest0-21swapper/023:07:450
5961992321,1cyclictest0-21swapper/022:52:480
5961992320,2cyclictest32021-21diskmemload21:25:560
596199230,2cyclictest2119-21sh00:04:550
596199230,23cyclictest0-21swapper/022:21:450
596199230,23cyclictest0-21swapper/021:12:380
596199230,22cyclictest7055-21ssh21:45:430
596199230,22cyclictest6754-21chrt22:41:240
596199230,22cyclictest23836-21apt-get22:00:480
596199230,22cyclictest2344-21sh00:33:170
596199230,22cyclictest0-21swapper/023:32:370
596199230,22cyclictest0-21swapper/021:10:000
596199230,0cyclictest32021-21diskmemload21:42:180
596199230,0cyclictest0-21swapper/022:06:340
596199230,0cyclictest0-21swapper/000:05:010
5974992222,0cyclictest0-21swapper/321:25:463
5974992222,0cyclictest0-21swapper/321:10:313
5974992220,1cyclictest20362-21munin-run19:40:003
597499220,22cyclictest0-21swapper/300:35:493
597499220,21cyclictest7150-21apt-get21:45:473
597499220,21cyclictest28900-21apt-get23:30:483
597499220,21cyclictest24087-21ssh23:55:333
597499220,21cyclictest20936-21ssh23:51:203
597499220,1cyclictest0-21swapper/322:55:583
597499220,1cyclictest0-21swapper/319:50:013
597499220,1cyclictest0-21swapper/319:50:013
597499220,1cyclictest0-21swapper/300:10:093
5969992222,0cyclictest0-21swapper/222:26:172
5969992221,1cyclictest0-21swapper/200:26:412
596999220,22cyclictest0-21swapper/222:08:162
596999220,22cyclictest0-21swapper/220:00:452
596999220,21cyclictest1351-21ssh21:40:392
596999220,21cyclictest10014-21awk20:25:002
596999220,21cyclictest0-21swapper/223:56:102
596999220,21cyclictest0-21swapper/221:21:192
596999220,21cyclictest0-21swapper/221:05:482
596999220,1cyclictest9827-21ssh23:41:272
596999220,1cyclictest2509-21ssh00:05:052
596999220,1cyclictest12129-21ssh21:50:052
596999220,1cyclictest0-21swapper/222:35:112
596999220,0cyclictest0-21swapper/223:15:102
5965992222,0cyclictest0-21swapper/123:46:171
5965992220,1cyclictest9096-21rm22:15:511
5965992220,1cyclictest4230-21sh22:40:151
596599220,21cyclictest18776-21apt-get23:50:481
596599220,21cyclictest11065-21apt-get19:20:481
596599220,21cyclictest0-21swapper/123:55:511
596599220,21cyclictest0-21swapper/123:35:081
596599220,21cyclictest0-21swapper/123:10:031
596599220,21cyclictest0-21swapper/122:55:231
596599220,21cyclictest0-21swapper/120:10:471
596599220,21cyclictest0-21swapper/100:30:441
596599220,1cyclictest16348-21ssh00:16:291
596599220,1cyclictest0-21swapper/122:12:571
5961992220,1cyclictest4790-21latency_hist20:15:000
596199220,21cyclictest32021-21diskmemload21:20:280
596199220,21cyclictest22041-21apt-get20:50:010
596199220,21cyclictest14807-21dump-pmu-power20:35:010
596199220,21cyclictest0-21swapper/020:35:470
596199220,1cyclictest32665-21apt-get20:05:480
596199220,1cyclictest32351-21apt20:05:010
596199220,1cyclictest0-21swapper/023:40:220
596199220,1cyclictest0-21swapper/021:15:110
596199220,1cyclictest0-21swapper/000:10:230
5974992121,0cyclictest10311-21apt-get20:25:473
5974992120,0cyclictest0-21swapper/320:30:483
5974992119,1cyclictest13137-21ls19:25:003
597499210,21cyclictest0-21swapper/323:20:093
597499210,21cyclictest0-21swapper/323:20:093
597499210,20cyclictest6252-21apt-get19:10:493
597499210,1cyclictest32055-21basename21:10:003
597499210,0cyclictest0-21swapper/320:50:463
596999210,21cyclictest0-21swapper/220:40:482
596999210,0cyclictest8642-21apt-get19:15:482
596999210,0cyclictest0-21swapper/200:10:282
5965992121,0cyclictest32021-21diskmemload21:45:111
5965992120,1cyclictest5095-21apt-get20:15:471
5965992120,0cyclictest0-21swapper/119:40:481
5965992119,1cyclictest22055-21dump-pmu-power20:50:011
5965992119,1cyclictest0-21swapper/120:00:011
596599210,20cyclictest0-21swapper/119:50:461
5961992121,0cyclictest0-21swapper/021:00:470
5961992120,1cyclictest0-21swapper/019:45:470
5961992120,1cyclictest0-21swapper/019:45:470
5961992119,1cyclictest11983-21kworker/u8:219:35:000
596199210,20cyclictest0-21swapper/020:25:470
5974992020,0cyclictest0-21swapper/321:00:473
5974992020,0cyclictest0-21swapper/320:35:473
5974992020,0cyclictest0-21swapper/319:25:483
5969992020,0cyclictest13461-21apt-get19:25:482
5969992020,0cyclictest12712-21apt-get20:30:482
5969992020,0cyclictest0-21swapper/220:50:022
5969992020,0cyclictest0-21swapper/220:15:472
5969992020,0cyclictest0-21swapper/219:55:482
596999200,2cyclictest13159-21cut19:25:002
5965992020,0cyclictest0-21swapper/120:30:481
5965992020,0cyclictest0-21swapper/119:10:491
596599200,0cyclictest0-21swapper/120:25:471
596199202,18cyclictest17901-21kworker/0:120:42:330
5961992020,0cyclictest22753-21apt-get20:50:460
5961992020,0cyclictest0-21swapper/020:55:480
5961992020,0cyclictest0-21swapper/019:15:480
5961992019,0cyclictest0-21swapper/019:20:480
5974991918,0cyclictest0-21swapper/320:45:483
5969991919,0cyclictest0-21swapper/220:55:482
5969991918,1cyclictest20670-21apt-get19:40:482
5969991917,1cyclictest0-21swapper/220:15:002
596999190,18cyclictest12401-21perf20:30:002
596999190,18cyclictest0-21swapper/220:45:482
5965991919,0cyclictest0-21swapper/121:05:481
5965991919,0cyclictest0-21swapper/120:05:481
5965991917,1cyclictest19596-21ls20:45:011
5965991917,1cyclictest17953-21awk19:35:001
5965991917,1cyclictest0-21swapper/120:55:001
596599190,18cyclictest0-21swapper/120:20:021
5961991919,0cyclictest0-21swapper/020:20:020
5961991915,2cyclictest313-21systemd-journal20:20:000
5974991818,0cyclictest0-21swapper/319:40:483
5974991818,0cyclictest0-21swapper/319:15:483
5969991818,0cyclictest0-21swapper/221:00:472
5969991818,0cyclictest0-21swapper/219:45:472
5969991818,0cyclictest0-21swapper/219:45:472
596599180,17cyclictest32362-21perf20:05:011
5961991815,2cyclictest27540-21idleruntime-cro19:55:010
5974991717,0cyclictest0-21swapper/320:20:483
597499170,16cyclictest27603-21perf19:55:013
5969991717,0cyclictest0-21swapper/219:35:482
5965991717,0cyclictest27571-21apt-get21:00:471
5961991717,0cyclictest0-21swapper/019:55:480
5974991616,0cyclictest0-21swapper/320:05:483
5974991614,1cyclictest7207-21perf20:20:003
597499160,1cyclictest0-21swapper/320:45:013
597499160,1cyclictest0-21swapper/320:10:513
597499160,15cyclictest30015-21munin-run20:00:013
597499160,15cyclictest27251-21perf21:00:003
5969991616,0cyclictest0-21swapper/219:10:492
5969991614,1cyclictest16152-21ntp_states20:35:522
596999160,16cyclictest0-21swapper/220:05:482
5965991616,0cyclictest0-21swapper/119:35:481
5965991616,0cyclictest0-21swapper/119:25:481
5961991616,0cyclictest0-21swapper/019:35:480
5961991616,0cyclictest0-21swapper/019:25:480
5961991614,1cyclictest0-21swapper/019:45:010
596199160,15cyclictest7300-21ntp_states19:10:540
5969991514,1cyclictest25457-21apt-get19:50:462
5974991414,0cyclictest0-21swapper/320:00:573
597499140,0cyclictest0-21swapper/319:30:293
596999140,0cyclictest0-21swapper/219:30:092
596599140,0cyclictest0-21swapper/119:45:151
596599140,0cyclictest0-21swapper/119:45:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional