You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-08 - 11:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri May 08, 2026 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3821992725,0cyclictest151rcu_preempt22:43:243
3821992624,0cyclictest151rcu_preempt22:24:093
3819992522,2cyclictest29800-21ssh00:29:552
381999250,25cyclictest0-21swapper/223:58:472
381999250,24cyclictest0-21swapper/222:26:032
381599250,24cyclictest3617-21ssh22:11:491
381599250,24cyclictest2947-21ssh21:42:481
381599250,24cyclictest0-21swapper/100:04:481
3811992523,1cyclictest0-21swapper/000:29:100
381199252,22cyclictest62550irq/127-eno121:18:420
381199250,24cyclictest29413-21diskmemload23:14:430
381199250,24cyclictest29413-21diskmemload22:27:390
381199250,24cyclictest16848-21ssh23:49:380
381199250,24cyclictest16848-21ssh23:49:380
381199250,24cyclictest0-21swapper/022:59:120
381199250,0cyclictest0-21swapper/000:14:150
3821992423,0cyclictest34-21ksoftirqd/300:27:123
3821992422,1cyclictest7298-21cron00:10:003
3821992422,1cyclictest12413-21sh23:45:533
3821992422,1cyclictest12413-21sh23:45:533
3821992421,2cyclictest8089-21rm23:12:533
3821992421,2cyclictest3302-21ssh23:09:323
3821992421,2cyclictest13398-21runrttasks00:15:113
382199240,24cyclictest0-21swapper/321:31:103
382199240,23cyclictest20955-21kernelversion22:56:383
382199240,23cyclictest0-21swapper/319:36:333
382199240,23cyclictest0-21swapper/300:38:193
382199240,0cyclictest29413-21diskmemload00:32:213
3819992422,1cyclictest0-21swapper/200:37:112
3819992419,2cyclictest0-21swapper/221:25:092
381999240,23cyclictest20221-1kworker/u9:222:21:592
381999240,1cyclictest10223-21ssh23:15:222
3815992421,2cyclictest27334-21sh23:58:301
381599240,2cyclictest4695-21ssh23:38:531
381599240,24cyclictest0-21swapper/123:02:421
381599240,23cyclictest30435-21ssh00:30:241
381599240,23cyclictest29413-21diskmemload22:37:391
381599240,23cyclictest21511-21sh23:53:151
381599240,23cyclictest16770-21ssh22:24:221
381599240,22cyclictest313-21systemd-journal20:05:011
381599240,1cyclictest29413-21diskmemload22:56:081
381599240,0cyclictest29413-21diskmemload21:31:231
381599240,0cyclictest0-21swapper/122:52:051
3811992423,1cyclictest62550irq/127-eno100:16:250
3811992423,0cyclictest0-21swapper/020:01:320
381199240,24cyclictest29413-21diskmemload21:57:220
381199240,23cyclictest0-21swapper/000:38:370
381199240,22cyclictest24892-21sh23:27:460
381199240,1cyclictest0-21swapper/021:35:040
381199240,0cyclictest0-21swapper/022:37:550
3821992323,0cyclictest0-21swapper/322:16:483
3821992322,1cyclictest29413-21diskmemload00:00:433
3821992322,1cyclictest1208-21apt-get23:36:343
3821992321,1cyclictest23316-21sh23:55:213
3821992321,1cyclictest0-21swapper/322:09:213
3821992320,2cyclictest26617-21ssh22:32:273
3821992320,2cyclictest18374-21sh22:26:143
382199230,22cyclictest25722-21ssh23:28:483
382199230,22cyclictest0-21swapper/323:31:433
382199230,22cyclictest0-21swapper/321:59:463
382199230,0cyclictest0-21swapper/323:16:413
382199230,0cyclictest0-21swapper/322:35:043
382199230,0cyclictest0-21swapper/321:20:533
3819992323,0cyclictest0-21swapper/223:45:192
3819992323,0cyclictest0-21swapper/223:45:192
3819992323,0cyclictest0-21swapper/223:13:082
3819992323,0cyclictest0-21swapper/222:38:102
3819992323,0cyclictest0-21swapper/221:34:282
3819992323,0cyclictest0-21swapper/200:05:082
3819992323,0cyclictest0-21swapper/200:00:512
3819992322,1cyclictest29413-21diskmemload21:35:222
3819992320,2cyclictest30351-21ssh21:10:512
3819992319,3cyclictest0-21swapper/222:55:192
381999230,23cyclictest29413-21diskmemload21:17:102
381999230,23cyclictest0-21swapper/200:21:072
381999230,22cyclictest528-21awk20:10:002
381999230,22cyclictest5043-21ssh21:45:192
381999230,22cyclictest4981-21sed22:41:442
381999230,22cyclictest29413-21diskmemload00:30:292
381999230,22cyclictest17094-21ssh22:52:482
381999230,22cyclictest13960-21ssh21:52:292
381999230,22cyclictest0-21swapper/223:30:052
381999230,22cyclictest0-21swapper/200:12:422
381999230,0cyclictest29413-21diskmemload23:36:222
381999230,0cyclictest29413-21diskmemload23:29:022
381999230,0cyclictest0-21swapper/221:58:302
381999230,0cyclictest0-21swapper/221:41:082
3815992322,1cyclictest12469-21ssh21:22:541
3815992321,1cyclictest29413-21diskmemload21:53:391
3815992321,1cyclictest0-21swapper/121:57:511
3815992320,2cyclictest12189-21sh22:48:461
381599230,2cyclictest24291-21latency_hist00:25:011
381599230,2cyclictest13992-21sh00:16:071
381599230,23cyclictest0-21swapper/100:28:261
381599230,22cyclictest4930-21ssh23:10:541
381599230,22cyclictest0-21swapper/123:26:091
381599230,22cyclictest0-21swapper/121:11:381
381599230,0cyclictest29413-21diskmemload23:09:221
381599230,0cyclictest29413-21diskmemload21:47:011
381599230,0cyclictest0-21swapper/122:07:511
3811992321,1cyclictest9327-21ssh22:16:590
3811992321,1cyclictest28978-21ssh00:00:190
3811992321,1cyclictest0-21swapper/021:44:340
3811992320,2cyclictest0-21swapper/023:32:200
3811992320,2cyclictest0-21swapper/021:51:100
3811992319,3cyclictest29197-21sh23:03:190
381199231,21cyclictest0-21swapper/023:50:020
381199231,1cyclictest62550irq/127-eno121:30:370
381199231,1cyclictest62550irq/127-eno100:20:180
381199230,2cyclictest31186-21sh23:05:480
381199230,23cyclictest0-21swapper/022:43:550
381199230,23cyclictest0-21swapper/022:08:250
381199230,22cyclictest3385-21apt-get20:16:330
381199230,22cyclictest29413-21diskmemload21:10:230
381199230,22cyclictest18588-21grep19:40:000
381199230,22cyclictest15342-21latency_hist23:20:010
381199230,22cyclictest0-21swapper/022:21:320
381199230,22cyclictest0-21swapper/021:06:310
381199230,1cyclictest0-21swapper/023:23:380
381199230,0cyclictest29413-21diskmemload23:37:490
381199230,0cyclictest29413-21diskmemload00:05:320
3821992222,0cyclictest29413-21diskmemload21:28:043
3821992222,0cyclictest0-21swapper/322:11:183
3821992222,0cyclictest0-21swapper/321:50:253
3821992222,0cyclictest0-21swapper/321:44:013
3821992221,1cyclictest25022-21apt-get21:01:343
3821992221,1cyclictest0-21swapper/322:45:273
3821992220,1cyclictest30330-21ssh21:10:493
3821992220,1cyclictest20986-21latency_hist19:45:013
3821992220,1cyclictest0-21swapper/321:15:263
3821992220,1cyclictest0-21swapper/320:45:013
3821992220,1cyclictest0-21swapper/320:01:333
3821992219,2cyclictest29413-21diskmemload22:00:583
3821992219,2cyclictest17823-21sh23:50:323
382199220,22cyclictest0-21swapper/322:51:373
382199220,21cyclictest29676-21sh23:04:063
382199220,21cyclictest0-21swapper/323:22:463
382199220,1cyclictest10437-21sh23:44:033
382199220,0cyclictest0-21swapper/300:10:293
3819992222,0cyclictest0-21swapper/221:06:322
3819992222,0cyclictest0-21swapper/200:19:182
3819992220,1cyclictest16179-21munin-run19:35:002
3819992220,1cyclictest11558-21awk22:20:012
3819992220,1cyclictest0-21swapper/223:51:002
3819992220,1cyclictest0-21swapper/219:50:012
3819992219,2cyclictest5430-21perf20:20:002
381999220,22cyclictest0-21swapper/222:02:522
381999220,21cyclictest0-21swapper/223:05:252
381999220,21cyclictest0-21swapper/222:10:122
381999220,21cyclictest0-21swapper/221:20:532
381999220,1cyclictest5757-21ssh23:40:032
381999220,1cyclictest24334-21apt-get23:00:022
381999220,1cyclictest16684-21apt-get23:21:342
381999220,0cyclictest0-21swapper/222:45:452
381999220,0cyclictest0-21swapper/222:31:002
3815992222,0cyclictest0-21swapper/123:21:151
3815992222,0cyclictest0-21swapper/122:40:201
3815992222,0cyclictest0-21swapper/121:15:211
3815992222,0cyclictest0-21swapper/100:10:451
3815992221,1cyclictest24777-21ssh22:01:531
3815992220,1cyclictest17577-21ssh22:25:131
381599220,22cyclictest0-21swapper/122:16:391
381599220,21cyclictest27891-21if_enp1s021:36:381
381599220,21cyclictest0-21swapper/120:40:011
381599220,21cyclictest0-21swapper/120:40:011
381599220,1cyclictest23537-21sh22:30:401
381599220,1cyclictest0-21swapper/123:41:351
381599220,1cyclictest0-21swapper/123:19:451
381599220,0cyclictest0-21swapper/123:30:461
381599220,0cyclictest0-21swapper/121:25:221
381599220,0cyclictest0-21swapper/100:35:181
381599220,0cyclictest0-21swapper/100:05:531
381199222,20cyclictest62550irq/127-eno123:40:310
3811992222,0cyclictest0-21swapper/022:51:100
3811992222,0cyclictest0-21swapper/022:00:370
3811992220,1cyclictest13362-21apt-get19:30:020
381199220,21cyclictest8084-21sh22:45:320
381199220,21cyclictest0-21swapper/022:30:030
381199220,21cyclictest0-21swapper/021:20:390
381199220,21cyclictest0-21swapper/020:15:000
381199220,1cyclictest783-21snmpd21:27:290
381199220,1cyclictest5090-21ssh22:13:490
381199220,1cyclictest4839-21ssh21:45:030
3821992121,0cyclictest0-21swapper/320:36:333
3821992121,0cyclictest0-21swapper/320:36:333
3821992120,1cyclictest27411-21apt-get21:06:323
382199210,1cyclictest26144-21apt-get19:56:323
382199210,0cyclictest0-21swapper/319:51:343
3819992120,1cyclictest8960-21apt-get19:21:342
3819992120,0cyclictest11354-21apt-get19:26:342
3819992119,1cyclictest18963-21apt-get19:41:352
381999210,21cyclictest8573-21perf19:20:002
381999210,0cyclictest0-21swapper/220:46:342
3815992121,0cyclictest0-21swapper/119:41:351
3815992119,1cyclictest18576-21sendmail-msp19:40:001
381599210,20cyclictest0-21swapper/121:06:311
3821992019,1cyclictest4150-21apt-get19:11:363
382199200,0cyclictest0-21swapper/320:16:333
382199200,0cyclictest0-21swapper/320:06:333
3819992020,0cyclictest0-21swapper/221:01:342
3819992018,1cyclictest10230-21perf20:30:002
381999200,20cyclictest0-21swapper/219:36:332
381999200,0cyclictest0-21swapper/220:35:012
381999200,0cyclictest0-21swapper/219:56:322
3815992020,0cyclictest0-21swapper/121:01:341
3815992020,0cyclictest0-21swapper/120:16:321
3815992018,1cyclictest0-21swapper/120:11:351
381599200,0cyclictest0-21swapper/120:31:321
3811992020,0cyclictest0-21swapper/020:26:340
3811992020,0cyclictest0-21swapper/020:06:330
3811992020,0cyclictest0-21swapper/019:26:330
3811992019,1cyclictest0-21swapper/021:01:340
3811992018,1cyclictest0-21swapper/020:41:340
381199200,19cyclictest12998-21apt-get20:36:320
381199200,19cyclictest12998-21apt-get20:36:320
3821991919,0cyclictest0-21swapper/320:31:333
3821991919,0cyclictest0-21swapper/320:26:333
3821991919,0cyclictest0-21swapper/319:31:333
3819991917,1cyclictest0-21swapper/220:55:012
3815991919,0cyclictest0-21swapper/119:56:321
3815991917,1cyclictest23751-21apt-get19:51:341
3811991917,1cyclictest20996-21munin-run19:45:010
381199190,18cyclictest17780-21/usr/sbin/munin20:46:390
3821991818,0cyclictest0-21swapper/319:21:343
3821991818,0cyclictest0-21swapper/319:16:343
3821991816,1cyclictest0-21swapper/320:51:333
3815991816,1cyclictest20207-21apt-get20:51:331
3811991818,0cyclictest0-21swapper/020:31:330
3811991818,0cyclictest0-21swapper/019:46:330
3811991818,0cyclictest0-21swapper/019:16:340
3821991717,0cyclictest0-21swapper/320:56:333
3821991717,0cyclictest0-21swapper/320:21:333
3821991716,0cyclictest0-21swapper/319:26:333
3815991716,1cyclictest8207-21apt-get20:26:331
3815991716,0cyclictest13992-21apt-get19:30:021
3811991717,0cyclictest0-21swapper/019:56:320
3819991616,0cyclictest0-21swapper/220:01:322
381999160,2cyclictest14643-21grep20:36:422
381999160,2cyclictest14643-21grep20:36:422
3815991616,0cyclictest0-21swapper/120:06:331
3811991616,0cyclictest0-21swapper/019:51:330
3811991615,0cyclictest5814-21apt-get20:21:340
3819991513,1cyclictest0-21swapper/220:45:012
3819991513,1cyclictest0-21swapper/219:55:012
3819991512,2cyclictest6008-21cpuspeed_turbos20:21:342
381999150,14cyclictest313-21systemd-journal21:00:002
3815991514,1cyclictest0-21swapper/119:46:331
3811991515,0cyclictest0-21swapper/020:51:330
3811991515,0cyclictest0-21swapper/019:11:360
3821991414,0cyclictest0-21swapper/320:47:063
3821991414,0cyclictest0-21swapper/320:10:063
382199140,14cyclictest0-21swapper/319:45:163
3819991414,0cyclictest0-21swapper/219:11:262
381999140,14cyclictest0-21swapper/220:10:312
381599140,14cyclictest0-21swapper/120:46:191
381599140,14cyclictest0-21swapper/119:15:141
381599140,0cyclictest0-21swapper/120:55:131
381599140,0cyclictest0-21swapper/120:40:151
381599140,0cyclictest0-21swapper/119:20:291
381599140,0cyclictest0-21swapper/119:10:541
3811991414,0cyclictest0-21swapper/019:20:050
381199140,0cyclictest0-21swapper/020:55:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional