You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 15:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Fri May 01, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5977992725,1cyclictest14605-21sh11:29:432
5977992725,1cyclictest14605-21sh11:29:432
598199250,24cyclictest0-21swapper/311:57:373
597799251,23cyclictest606-21dbus-daemon12:05:002
597199250,25cyclictest463-21diskmemload09:30:210
597199250,24cyclictest5458-21apt-get11:21:000
597199250,24cyclictest463-21diskmemload11:18:030
597199250,24cyclictest463-21diskmemload10:33:060
597199250,24cyclictest13177-21cron10:25:000
597199250,1cyclictest62550irq/127-eno112:05:160
598199240,2cyclictest22721-21sh11:36:313
598199240,23cyclictest463-21diskmemload12:31:173
598199240,23cyclictest0-21swapper/312:27:123
598199240,23cyclictest0-21swapper/312:23:313
598199240,1cyclictest25658-21apt-get10:06:013
597799240,23cyclictest0-21swapper/210:11:102
597799240,1cyclictest0-21swapper/212:32:252
5975992424,0cyclictest0-21swapper/111:19:511
5975992423,1cyclictest2051-21apt-get10:45:581
597599240,23cyclictest463-21diskmemload11:23:501
597599240,23cyclictest463-21diskmemload11:10:391
597599240,23cyclictest463-21diskmemload10:00:591
597599240,0cyclictest31904-21/usr/sbin/munin10:11:071
5971992421,2cyclictest3392-21sh10:15:190
597199240,24cyclictest62550irq/127-eno112:37:530
597199240,24cyclictest0-21swapper/009:46:360
597199240,23cyclictest463-21diskmemload10:57:440
597199240,23cyclictest10117-21ssh09:19:010
597199240,0cyclictest463-21diskmemload12:19:130
597199240,0cyclictest463-21diskmemload11:49:230
5981992323,0cyclictest463-21diskmemload12:05:103
5981992323,0cyclictest0-21swapper/311:40:463
5981992323,0cyclictest0-21swapper/310:43:383
5981992323,0cyclictest0-21swapper/309:47:283
5981992322,0cyclictest0-21swapper/309:16:183
5981992321,1cyclictest0-21swapper/312:12:163
5981992321,1cyclictest0-21swapper/311:29:523
5981992321,1cyclictest0-21swapper/311:29:523
5981992321,1cyclictest0-21swapper/310:34:133
5981992321,1cyclictest0-21swapper/310:22:543
5981992321,1cyclictest0-21swapper/309:21:093
5981992320,2cyclictest17682-21ssh09:57:323
598199230,23cyclictest0-21swapper/311:01:453
598199230,23cyclictest0-21swapper/309:52:303
598199230,22cyclictest463-21diskmemload10:01:333
598199230,22cyclictest2352-21ssh10:14:433
598199230,22cyclictest13579-21ssh10:25:153
598199230,22cyclictest0-21swapper/312:37:423
598199230,22cyclictest0-21swapper/310:50:413
598199230,22cyclictest0-21swapper/310:16:273
598199230,22cyclictest0-21swapper/309:13:173
598199230,0cyclictest463-21diskmemload10:39:073
598199230,0cyclictest0-21swapper/312:19:093
598199230,0cyclictest0-21swapper/312:02:053
598199230,0cyclictest0-21swapper/311:45:183
598199230,0cyclictest0-21swapper/311:33:133
598199230,0cyclictest0-21swapper/311:22:053
5977992323,0cyclictest0-21swapper/211:51:482
5977992323,0cyclictest0-21swapper/211:12:502
5977992323,0cyclictest0-21swapper/210:04:302
5977992323,0cyclictest0-21swapper/209:56:472
5977992321,1cyclictest7412-21apt-get12:26:002
5977992321,1cyclictest22496-21ssh11:06:002
5977992320,2cyclictest463-21diskmemload09:31:022
5977992320,2cyclictest20710-21ssh11:35:562
5977992320,2cyclictest11995-21ssh10:55:322
597799230,23cyclictest0-21swapper/210:41:112
597799230,22cyclictest8701-21rm10:20:402
597799230,22cyclictest463-21diskmemload10:33:102
597799230,22cyclictest463-21diskmemload09:44:432
597799230,22cyclictest463-21diskmemload09:35:502
597799230,22cyclictest30923-21/usr/sbin/munin11:46:072
597799230,22cyclictest27695-21perf09:00:002
597799230,22cyclictest17424-21apt-get11:01:012
597799230,22cyclictest0-21swapper/209:26:582
597799230,1cyclictest0-21swapper/210:46:482
597799230,0cyclictest0-21swapper/212:12:162
597799230,0cyclictest0-21swapper/212:08:532
597799230,0cyclictest0-21swapper/210:36:522
597799230,0cyclictest0-21swapper/209:48:462
597799230,0cyclictest0-21swapper/209:17:222
5975992323,0cyclictest0-21swapper/112:35:131
5975992323,0cyclictest0-21swapper/111:48:181
5975992323,0cyclictest0-21swapper/111:25:231
5975992323,0cyclictest0-21swapper/111:25:231
5975992323,0cyclictest0-21swapper/109:15:471
5975992321,1cyclictest19314-21expr12:06:021
5975992320,2cyclictest9825-21cut10:21:051
597599231,21cyclictest0-21swapper/112:23:011
597599230,23cyclictest0-21swapper/112:25:031
597599230,23cyclictest0-21swapper/112:16:301
597599230,23cyclictest0-21swapper/109:30:561
597599230,22cyclictest6576-21diskmemload10:17:331
597599230,22cyclictest0-21swapper/111:06:011
597599230,0cyclictest463-21diskmemload10:35:061
597599230,0cyclictest463-21diskmemload09:27:181
597599230,0cyclictest0-21swapper/112:03:101
597599230,0cyclictest0-21swapper/110:59:431
5971992323,0cyclictest0-21swapper/012:31:310
5971992323,0cyclictest0-21swapper/012:12:470
5971992323,0cyclictest0-21swapper/010:40:590
5971992323,0cyclictest0-21swapper/009:10:310
5971992321,2cyclictest0-21swapper/009:25:090
5971992321,1cyclictest2145-21ssh12:20:530
5971992321,1cyclictest0-21swapper/011:25:590
5971992321,1cyclictest0-21swapper/011:25:590
597199230,2cyclictest8983-21expr12:26:100
597199230,23cyclictest0-21swapper/010:51:160
597199230,23cyclictest0-21swapper/010:05:280
597199230,22cyclictest14899-21ssh09:55:520
597199230,22cyclictest12220-21ssh09:51:590
597199230,22cyclictest11423-21ssh09:20:430
597199230,1cyclictest0-21swapper/009:45:010
597199230,0cyclictest0-21swapper/011:40:500
597199230,0cyclictest0-21swapper/011:06:020
597199230,0cyclictest0-21swapper/010:25:030
5981992222,0cyclictest0-21swapper/311:07:253
5981992220,1cyclictest0-21swapper/310:46:343
5981992220,1cyclictest0-21swapper/309:36:103
5981992220,1cyclictest0-21swapper/308:35:013
5981992219,2cyclictest30073-21wc08:00:003
598199220,2cyclictest11834-21rm10:55:183
598199220,22cyclictest0-21swapper/309:40:033
598199220,21cyclictest463-21diskmemload11:16:013
598199220,21cyclictest21891-21apt-get09:31:013
598199220,21cyclictest0-21swapper/311:10:313
598199220,21cyclictest0-21swapper/308:45:023
598199220,21cyclictest0-21swapper/308:35:023
598199220,21cyclictest0-21swapper/308:25:013
5977992222,0cyclictest0-21swapper/212:21:052
5977992222,0cyclictest0-21swapper/210:07:182
5977992220,1cyclictest13424-21grep09:21:092
5977992220,1cyclictest0-21swapper/212:37:012
5977992220,1cyclictest0-21swapper/211:31:042
597799220,22cyclictest2576-21apt-get10:15:022
597799220,21cyclictest0-21swapper/212:15:272
597799220,21cyclictest0-21swapper/210:27:232
597799220,21cyclictest0-21swapper/209:50:412
597799220,1cyclictest330-21apt-get11:16:012
597799220,1cyclictest17297-21apt-get08:40:022
597799220,1cyclictest0-21swapper/210:53:042
597799220,0cyclictest0-21swapper/211:41:352
597799220,0cyclictest0-21swapper/211:24:182
5975992222,0cyclictest463-21diskmemload09:46:451
5975992222,0cyclictest0-21swapper/109:41:091
5975992222,0cyclictest0-21swapper/109:37:041
5975992221,1cyclictest15348-21ssh12:32:531
5975992221,1cyclictest0-21swapper/111:55:311
597599221,21cyclictest18686-21ssh09:59:021
597599220,22cyclictest0-21swapper/111:36:551
597599220,21cyclictest463-21diskmemload11:51:581
597599220,21cyclictest29273-21sh10:40:551
597599220,21cyclictest23231-21apt-get08:51:001
597599220,21cyclictest18055-21apt-get08:41:011
597599220,21cyclictest13991-21apt-get10:25:591
597599220,21cyclictest11102-21apt-get07:21:001
597599220,21cyclictest0-21swapper/109:10:021
597599220,21cyclictest0-21swapper/108:16:011
597599220,1cyclictest9934-21apt-get09:50:591
597599220,1cyclictest21788-21ssh10:32:271
597599220,1cyclictest0-21swapper/108:56:091
597599220,0cyclictest0-21swapper/109:20:421
5971992222,0cyclictest0-21swapper/010:01:550
597199220,22cyclictest0-21swapper/010:11:010
597199220,21cyclictest463-21diskmemload11:35:470
597199220,21cyclictest463-21diskmemload10:46:290
597199220,21cyclictest463-21diskmemload09:35:480
597199220,21cyclictest2839-21apt-get08:11:010
597199220,21cyclictest27074-21ssh11:10:110
597199220,21cyclictest10754-21apt07:20:000
597199220,21cyclictest0-21swapper/011:55:150
597199220,21cyclictest0-21swapper/011:51:120
597199220,1cyclictest24120-21ssh10:35:500
597199220,1cyclictest0-21swapper/011:00:310
597199220,0cyclictest18286-21apt-get07:35:590
5981992119,1cyclictest30886-21apt-get09:06:003
598199210,20cyclictest0-21swapper/309:26:483
598199210,20cyclictest0-21swapper/308:26:003
598199210,20cyclictest0-21swapper/307:30:593
598199210,20cyclictest0-21swapper/307:15:593
5977992121,0cyclictest0-21swapper/209:06:002
5977992120,0cyclictest0-21swapper/207:35:592
5977992119,1cyclictest606-21dbus-daemon08:25:012
5977992119,1cyclictest30462-21apt-get08:01:012
597799210,21cyclictest0-21swapper/209:10:292
597799210,20cyclictest0-21swapper/208:36:012
597799210,1cyclictest0-21swapper/208:45:022
597799210,0cyclictest0-21swapper/208:06:012
5975992121,0cyclictest0-21swapper/111:30:391
5975992121,0cyclictest0-21swapper/110:05:151
5975992120,1cyclictest0-21swapper/107:26:011
5975992120,0cyclictest0-21swapper/107:30:591
5975992119,1cyclictest9723-21perf08:25:011
5975992119,1cyclictest30527-21cron09:05:011
597599210,21cyclictest0-21swapper/110:50:041
597599210,21cyclictest0-21swapper/107:35:591
597599210,20cyclictest8678-21apt-get07:15:591
597599210,20cyclictest12456-21apt-get08:31:001
597599210,20cyclictest0-21swapper/108:36:011
597599210,0cyclictest10058-21apt-get08:26:001
597599210,0cyclictest0-21swapper/111:01:061
597599210,0cyclictest0-21swapper/107:46:001
5971992121,0cyclictest0-21swapper/008:41:010
5971992121,0cyclictest0-21swapper/008:16:010
5971992120,1cyclictest0-21swapper/008:50:000
5971992120,0cyclictest0-21swapper/007:26:010
5971992119,1cyclictest13156-21latency_hist07:25:000
597199210,0cyclictest0-21swapper/007:50:590
5981992020,0cyclictest0-21swapper/308:01:013
598199200,20cyclictest0-21swapper/307:41:013
598199200,0cyclictest25593-21apt-get07:50:583
5977992019,1cyclictest0-21swapper/208:11:012
597799200,20cyclictest0-21swapper/207:21:002
597799200,0cyclictest0-21swapper/208:25:592
597799200,0cyclictest0-21swapper/207:56:002
5971992020,0cyclictest0-21swapper/009:06:000
5971992020,0cyclictest0-21swapper/009:00:030
5971992020,0cyclictest0-21swapper/008:31:010
5971992020,0cyclictest0-21swapper/007:41:010
5971992018,1cyclictest17950-21latency_hist07:35:010
5981991919,0cyclictest0-21swapper/307:46:003
5981991919,0cyclictest0-21swapper/307:26:013
5981991918,0cyclictest5236-21apt-get08:16:013
5981991918,0cyclictest0-21swapper/308:11:023
5981991917,1cyclictest30557-21turbostat.cron09:05:013
598199190,18cyclictest0-21swapper/308:56:023
5977991919,0cyclictest6287-21apt-get07:11:012
5977991919,0cyclictest0-21swapper/208:16:012
5977991919,0cyclictest0-21swapper/207:46:002
5975991919,0cyclictest27991-21apt-get07:56:001
5975991919,0cyclictest0-21swapper/108:11:021
597599190,18cyclictest0-21swapper/109:06:001
5971991919,0cyclictest0-21swapper/008:51:000
5981991818,0cyclictest0-21swapper/307:35:593
5977991814,3cyclictest22844-21cron07:45:012
5975991818,0cyclictest0-21swapper/107:41:001
5971991818,0cyclictest0-21swapper/007:46:000
5981991717,0cyclictest0-21swapper/308:41:013
5977991717,0cyclictest0-21swapper/207:15:592
5975991717,0cyclictest0-21swapper/108:01:011
5975991715,1cyclictest0-21swapper/107:50:591
5971991717,0cyclictest0-21swapper/008:25:590
5971991717,0cyclictest0-21swapper/008:21:000
5971991716,0cyclictest0-21swapper/007:11:010
5977991616,0cyclictest13497-21apt-get07:26:012
5977991616,0cyclictest0-21swapper/207:30:592
5971991616,0cyclictest0-21swapper/008:56:020
597199160,15cyclictest2486-21awk08:10:000
5977991515,0cyclictest0-21swapper/208:51:002
5977991513,1cyclictest28-21ksoftirqd/207:50:592
5977991513,1cyclictest0-21swapper/209:01:052
5977991512,2cyclictest14494-21cron08:35:012
5975991513,1cyclictest7336-21ntp_states07:11:071
597199150,1cyclictest16152-21memory08:36:050
598199140,14cyclictest0-21swapper/308:51:273
598199140,14cyclictest0-21swapper/308:05:433
598199140,0cyclictest0-21swapper/307:20:213
597199140,14cyclictest0-21swapper/007:56:220
597199140,0cyclictest0-21swapper/008:01:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional