You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 21:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Thu Apr 16, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2289992826,1cyclictest28-21ksoftirqd/210:35:002
2289992626,0cyclictest0-21swapper/210:46:582
2289992625,1cyclictest28-21ksoftirqd/209:27:072
228999261,24cyclictest151rcu_preempt09:49:562
2288992624,1cyclictest12481-21dump-pmu-power11:35:001
228799263,22cyclictest62550irq/127-eno112:29:590
229199250,2cyclictest32471-21sh12:27:533
2289992522,0cyclictest0-21swapper/212:30:212
228999250,2cyclictest281ksoftirqd/210:51:362
228999250,24cyclictest28825-21diskmemload11:34:192
228999250,24cyclictest0-21swapper/209:34:512
228899250,24cyclictest28825-21diskmemload11:38:201
228799250,24cyclictest28825-21diskmemload10:10:530
228799250,24cyclictest12478-21awk11:35:000
228799250,24cyclictest0-21swapper/011:40:450
228799250,23cyclictest313-21systemd-journal08:35:010
2291992422,1cyclictest0-21swapper/311:59:493
2291992421,2cyclictest10362-21sh11:00:203
229199240,23cyclictest7387-21sleep10:25:013
229199240,23cyclictest28825-21diskmemload12:10:073
229199240,23cyclictest11180-21dump-pmu-power08:35:013
229199240,23cyclictest0-21swapper/309:35:533
229199240,1cyclictest9708-21sh11:33:213
229199240,1cyclictest28825-21diskmemload10:56:133
228999240,23cyclictest28825-21diskmemload11:23:542
228999240,23cyclictest28825-21diskmemload09:50:292
228999240,23cyclictest1908-21ssh12:29:502
228999240,1cyclictest0-21swapper/211:53:132
228899240,23cyclictest17865-21taskset10:35:351
228899240,23cyclictest15714-21ssh12:10:381
228899240,23cyclictest14450-21apt-get09:29:461
228899240,1cyclictest0-21swapper/112:20:391
228899240,0cyclictest0-21swapper/109:11:371
2287992423,1cyclictest62550irq/127-eno111:39:070
228799240,24cyclictest0-21swapper/010:02:310
228799240,23cyclictest28825-21diskmemload11:00:570
228799240,23cyclictest0-21swapper/012:32:480
228799240,23cyclictest0-21swapper/012:20:490
228799240,0cyclictest0-21swapper/010:06:140
228799240,0cyclictest0-21swapper/009:34:080
2291992323,0cyclictest0-21swapper/311:22:003
2291992323,0cyclictest0-21swapper/310:12:163
2291992320,2cyclictest28965-21sshd09:43:583
229199230,2cyclictest31545-21ssh10:18:183
229199230,23cyclictest28825-21diskmemload11:27:193
229199230,23cyclictest28825-21diskmemload10:07:153
229199230,23cyclictest0-21swapper/312:04:213
229199230,22cyclictest25547-21ssh11:15:453
229199230,22cyclictest19907-21ssh10:38:563
229199230,22cyclictest19873-21df09:34:473
229199230,0cyclictest28825-21diskmemload09:45:463
229199230,0cyclictest0-21swapper/311:47:183
2289992323,0cyclictest28825-21diskmemload09:20:202
2289992323,0cyclictest0-21swapper/212:15:542
2289992323,0cyclictest0-21swapper/209:13:242
2289992321,1cyclictest0-21swapper/212:14:412
2289992321,1cyclictest0-21swapper/211:07:022
2289992321,1cyclictest0-21swapper/210:12:562
2289992321,0cyclictest0-21swapper/211:38:342
2289992321,0cyclictest0-21swapper/210:56:072
2289992320,2cyclictest24303-21cron10:10:012
2289992318,3cyclictest4658-21ssh10:23:242
228999230,2cyclictest6382-21sh08:25:012
228999230,23cyclictest0-21swapper/209:36:392
228999230,22cyclictest11651-21ssh09:58:372
228999230,22cyclictest0-21swapper/211:16:412
228999230,1cyclictest7886-21ssh10:25:272
228999230,1cyclictest28825-21diskmemload12:06:092
228999230,0cyclictest28825-21diskmemload10:39:512
228999230,0cyclictest0-21swapper/211:28:532
2288992323,0cyclictest0-21swapper/112:38:511
2288992323,0cyclictest0-21swapper/110:20:351
2288992323,0cyclictest0-21swapper/110:05:501
2288992323,0cyclictest0-21swapper/109:52:521
2288992323,0cyclictest0-21swapper/109:34:081
2288992321,1cyclictest0-21swapper/111:45:081
2288992321,1cyclictest0-21swapper/111:03:451
2288992321,1cyclictest0-21swapper/109:00:001
228899230,2cyclictest9902-21turbostat11:00:011
228899230,23cyclictest0-21swapper/112:29:351
228899230,23cyclictest0-21swapper/111:56:271
228899230,23cyclictest0-21swapper/111:43:131
228899230,22cyclictest7164-21sh09:20:521
228899230,22cyclictest29404-21munin-run10:15:011
228899230,22cyclictest26277-21ssh11:16:551
228899230,22cyclictest21586-21ssh12:17:131
228899230,22cyclictest12225-21sh09:59:311
228899230,1cyclictest24024-21ssh09:39:071
228899230,0cyclictest0-21swapper/112:04:381
228899230,0cyclictest0-21swapper/109:40:271
2287992323,0cyclictest28825-21diskmemload12:05:570
2287992323,0cyclictest28825-21diskmemload11:55:470
2287992323,0cyclictest28825-21diskmemload09:27:410
2287992323,0cyclictest0-21swapper/011:20:440
2287992323,0cyclictest0-21swapper/011:13:180
2287992323,0cyclictest0-21swapper/010:38:090
2287992322,1cyclictest62550irq/127-eno109:57:360
2287992322,1cyclictest0-21swapper/012:00:070
2287992322,1cyclictest0-21swapper/009:20:140
2287992321,1cyclictest16543-21sh12:12:050
228799230,23cyclictest0-21swapper/011:48:430
228799230,23cyclictest0-21swapper/010:50:310
228799230,22cyclictest8762-21ssh12:35:080
228799230,22cyclictest5665-21phc2sys09:19:510
228799230,22cyclictest32640-21ssh09:46:420
228799230,22cyclictest30949-21sh10:17:160
228799230,22cyclictest2633-21ssh10:20:070
228799230,22cyclictest23951-21cron09:00:000
228799230,22cyclictest21825-21ssh09:35:220
228799230,22cyclictest1407-21latency_hist09:15:010
228799230,1cyclictest32023-21ptp4l11:54:530
228799230,0cyclictest28825-21diskmemload09:52:520
228799230,0cyclictest0-21swapper/010:55:460
2291992222,0cyclictest0-21swapper/310:32:543
2291992222,0cyclictest0-21swapper/309:58:093
2291992222,0cyclictest0-21swapper/309:12:163
2291992221,1cyclictest0-21swapper/307:29:463
2291992220,1cyclictest0-21swapper/312:20:203
2291992220,1cyclictest0-21swapper/309:20:473
229199220,2cyclictest2889-21sh10:54:333
229199220,22cyclictest5391-21ssh09:51:243
229199220,21cyclictest4809-21sh12:31:493
229199220,21cyclictest28825-21diskmemload12:05:163
229199220,21cyclictest24006-21munin-run09:00:003
229199220,21cyclictest0-21swapper/311:35:173
229199220,21cyclictest0-21swapper/310:46:133
229199220,21cyclictest0-21swapper/310:02:323
229199220,21cyclictest0-21swapper/309:19:453
229199220,21cyclictest0-21swapper/308:14:453
229199220,1cyclictest11927-21ssh09:25:263
229199220,0cyclictest30686-21apt-get11:54:463
229199220,0cyclictest0-21swapper/312:36:233
229199220,0cyclictest0-21swapper/312:16:003
2289992222,0cyclictest28825-21diskmemload10:41:242
2289992222,0cyclictest0-21swapper/212:03:482
2289992222,0cyclictest0-21swapper/211:41:272
2289992220,1cyclictest313-21systemd-journal08:50:002
2289992220,1cyclictest27215-21ssh09:40:592
2289992220,1cyclictest0-21swapper/212:22:162
2289992220,1cyclictest0-21swapper/211:10:302
228999220,22cyclictest0-21swapper/212:36:442
228999220,22cyclictest0-21swapper/210:03:072
228999220,21cyclictest0-21swapper/210:15:302
228999220,21cyclictest0-21swapper/208:19:452
228999220,21cyclictest0-21swapper/208:04:452
228999220,21cyclictest0-21swapper/207:59:472
228999220,1cyclictest3040-21ssh11:59:322
228999220,1cyclictest0-21swapper/211:02:242
228999220,1cyclictest0-21swapper/209:18:062
228999220,0cyclictest0-21swapper/208:39:452
2288992222,0cyclictest28825-21diskmemload11:12:191
2288992222,0cyclictest0-21swapper/110:50:311
2288992222,0cyclictest0-21swapper/109:18:181
2288992221,0cyclictest0-21swapper/107:59:461
2288992220,1cyclictest11185-21latency_hist08:35:011
228899220,2cyclictest31685-21sh09:45:131
228899220,22cyclictest0-21swapper/112:30:381
228899220,22cyclictest0-21swapper/111:21:011
228899220,22cyclictest0-21swapper/110:40:181
228899220,21cyclictest7022-21awk07:20:001
228899220,21cyclictest28825-21diskmemload11:06:001
228899220,21cyclictest10065-21apt-get07:29:461
228899220,21cyclictest0-21swapper/110:45:191
228899220,21cyclictest0-21swapper/108:10:001
228899220,1cyclictest7484-21awk11:30:001
228899220,1cyclictest28825-21diskmemload11:50:091
228899220,1cyclictest28825-21diskmemload10:16:161
228899220,1cyclictest27067-21apt-get09:09:461
228899220,1cyclictest0-21swapper/112:08:271
2287992222,0cyclictest0-21swapper/011:05:400
2287992221,1cyclictest0-21swapper/012:16:530
2287992220,1cyclictest6397-21perf08:25:010
228799220,2cyclictest27542-21ssh09:41:360
228799220,21cyclictest774-21runrttasks09:09:450
228799220,21cyclictest28825-21diskmemload10:45:590
228799220,21cyclictest24958-21apt-get07:59:460
228799220,21cyclictest0-21swapper/010:31:170
228799220,21cyclictest0-21swapper/007:39:440
228799220,1cyclictest7761-21ssh10:25:160
228799220,1cyclictest2913-21ssh11:25:150
228799220,1cyclictest28825-21diskmemload10:44:070
2291992120,1cyclictest16382-21apt-get08:45:023
2291992120,0cyclictest0-21swapper/307:14:443
2291992119,1cyclictest7662-21ssh10:25:063
2291992119,1cyclictest606-21dbus-daemon07:20:003
2291992119,1cyclictest17376-21apt-get07:44:453
229199210,21cyclictest0-21swapper/310:40:153
229199210,1cyclictest20143-21ssh11:10:023
229199210,0cyclictest0-21swapper/311:40:273
229199210,0cyclictest0-21swapper/308:35:023
229199210,0cyclictest0-21swapper/307:54:443
2289992121,0cyclictest0-21swapper/211:45:272
2289992121,0cyclictest0-21swapper/208:09:452
2289992119,1cyclictest24003-21perf09:00:002
228999210,0cyclictest0-21swapper/209:04:452
228999210,0cyclictest0-21swapper/208:54:432
2288992121,0cyclictest0-21swapper/108:44:461
2288992120,1cyclictest0-21swapper/108:45:021
228899210,21cyclictest0-21swapper/110:25:161
228899210,0cyclictest0-21swapper/110:33:021
228899210,0cyclictest0-21swapper/107:44:441
2287992121,0cyclictest22168-21apt-get07:54:440
2287992120,1cyclictest12211-21apt-get08:39:450
228799210,20cyclictest7665-21apt-get07:24:440
228799210,1cyclictest31590-21turbostat08:10:000
228799210,0cyclictest0-21swapper/008:45:020
2291992020,0cyclictest0-21swapper/308:19:453
2291992020,0cyclictest0-21swapper/307:39:443
2291992019,1cyclictest14626-21apt-get08:44:463
2289992020,0cyclictest0-21swapper/207:44:452
2289992020,0cyclictest0-21swapper/207:39:442
228999200,0cyclictest0-21swapper/208:14:452
2288992019,0cyclictest14959-21apt-get07:39:441
2288992018,1cyclictest6414-21awk08:25:011
2287992020,0cyclictest0-21swapper/008:44:460
2291991919,0cyclictest0-21swapper/308:54:433
2291991919,0cyclictest0-21swapper/307:49:463
2289991919,0cyclictest0-21swapper/208:34:452
2289991919,0cyclictest0-21swapper/207:34:442
2289991918,0cyclictest2852-21apt-get07:14:442
2288991919,0cyclictest0-21swapper/108:04:451
2288991919,0cyclictest0-21swapper/107:24:441
228899190,18cyclictest0-21swapper/107:34:441
2287991919,0cyclictest19807-21apt-get08:54:430
2291991816,1cyclictest0-21swapper/309:10:003
229199180,17cyclictest0-21swapper/307:59:473
2289991818,0cyclictest271rcuc/207:29:472
2289991818,0cyclictest0-21swapper/207:24:442
2288991818,0cyclictest0-21swapper/107:54:441
2287991818,0cyclictest5247-21apt-get07:19:430
2287991817,0cyclictest2203-21apt-get08:19:450
2291991717,0cyclictest0-21swapper/308:29:463
2291991717,0cyclictest0-21swapper/308:24:463
2291991715,1cyclictest14231-21awk07:35:013
2289991717,0cyclictest0-21swapper/207:19:432
2289991715,1cyclictest0-21swapper/209:10:002
2288991717,0cyclictest0-21swapper/108:39:451
2287991717,0cyclictest0-21swapper/007:44:450
2287991717,0cyclictest0-21swapper/007:14:440
228799170,0cyclictest0-21swapper/009:04:450
2289991616,0cyclictest0-21swapper/208:44:462
2288991616,0cyclictest0-21swapper/109:04:451
2288991616,0cyclictest0-21swapper/108:54:431
2287991614,1cyclictest21559-21munin-run07:50:010
2291991515,0cyclictest27427-21apt-get08:04:453
2289991515,0cyclictest0-21swapper/207:54:442
2289991514,0cyclictest0-21swapper/207:49:462
2288991515,0cyclictest0-21swapper/108:19:451
228899150,1cyclictest0-21swapper/108:29:511
2287991515,0cyclictest0-21swapper/007:34:440
228799150,0cyclictest7020-21apt-get08:29:450
2291991414,0cyclictest0-21swapper/307:20:083
229199140,14cyclictest0-21swapper/308:05:243
229199140,0cyclictest0-21swapper/309:01:133
228999140,14cyclictest0-21swapper/208:25:372
2288991414,0cyclictest0-21swapper/107:45:291
228899140,0cyclictest0-21swapper/107:10:461
2287991414,0cyclictest0-21swapper/007:25:130
228799140,14cyclictest0-21swapper/008:10:160
228799140,14cyclictest0-21swapper/008:00:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional