You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 16:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Thu Jul 16, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
441992827,1cyclictest22-21ksoftirqd/110:20:411
441992827,1cyclictest22-21ksoftirqd/109:20:511
444992725,1cyclictest0-21swapper/211:50:002
45399250,24cyclictest26634-21diskmemload11:58:303
45399250,24cyclictest26634-21diskmemload10:05:403
45399250,24cyclictest0-21swapper/310:01:473
44499250,24cyclictest0-21swapper/211:57:382
439992523,2cyclictest12054-21diskmemload12:14:510
439992523,2cyclictest0-21swapper/011:50:220
453992422,1cyclictest0-21swapper/310:28:133
45399240,24cyclictest0-21swapper/310:23:293
45399240,23cyclictest19062-21apt-get08:55:033
45399240,23cyclictest0-21swapper/310:55:013
45399240,23cyclictest0-21swapper/310:51:093
45399240,23cyclictest0-21swapper/310:14:273
45399240,23cyclictest0-21swapper/309:31:073
45399240,1cyclictest10237-21apt-get11:17:183
45399240,0cyclictest26634-21diskmemload11:14:193
45399240,0cyclictest0-21swapper/312:20:043
45399240,0cyclictest0-21swapper/311:45:083
444992423,1cyclictest0-21swapper/212:02:122
44499240,24cyclictest0-21swapper/211:17:522
44499240,24cyclictest0-21swapper/209:21:562
44499240,23cyclictest26634-21diskmemload10:45:192
44499240,23cyclictest26634-21diskmemload09:37:242
44499240,23cyclictest0-21swapper/211:27:072
44499240,23cyclictest0-21swapper/210:55:392
44499240,21cyclictest1686-21ssh12:05:552
441992423,1cyclictest29925-21ssh10:08:581
441992423,1cyclictest26106-21ssh11:02:201
44199242,21cyclictest22-21ksoftirqd/111:51:341
441992422,1cyclictest13695-21/usr/sbin/munin11:47:251
44199240,24cyclictest0-21swapper/111:31:591
44199240,23cyclictest23846-21sh12:25:121
44199240,23cyclictest10019-21perf10:20:001
44199240,0cyclictest26634-21diskmemload09:10:211
44199240,0cyclictest151rcu_preempt10:35:011
44199240,0cyclictest0-21swapper/109:27:301
439992422,1cyclictest0-21swapper/010:02:280
439992421,2cyclictest29561-21ssh10:08:240
43999240,24cyclictest62550irq/127-eno109:16:020
43999240,24cyclictest0-21swapper/011:20:060
43999240,23cyclictest29085-21sh12:30:000
43999240,23cyclictest26634-21diskmemload12:00:480
43999240,23cyclictest26634-21diskmemload10:42:200
43999240,23cyclictest0-21swapper/012:09:060
43999240,23cyclictest0-21swapper/010:39:180
43999240,0cyclictest0-21swapper/009:36:220
453992323,0cyclictest0-21swapper/312:27:323
453992323,0cyclictest0-21swapper/311:24:493
453992323,0cyclictest0-21swapper/310:34:043
453992322,1cyclictest0-21swapper/312:14:203
453992321,1cyclictest0-21swapper/312:07:423
45399230,23cyclictest0-21swapper/311:53:583
45399230,23cyclictest0-21swapper/310:47:233
45399230,23cyclictest0-21swapper/309:12:163
45399230,22cyclictest4389-21sh11:40:003
45399230,22cyclictest29807-21sh11:33:363
45399230,22cyclictest19431-21turbostat.cron10:00:013
45399230,22cyclictest10090-21idleruntime-cro07:30:013
45399230,1cyclictest0-21swapper/312:31:173
45399230,0cyclictest0-21swapper/312:16:443
45399230,0cyclictest0-21swapper/312:01:333
45399230,0cyclictest0-21swapper/309:37:293
45399230,0cyclictest0-21swapper/309:28:403
444992323,0cyclictest0-21swapper/212:12:092
444992323,0cyclictest0-21swapper/211:38:002
444992323,0cyclictest0-21swapper/210:54:472
444992320,3cyclictest0-21swapper/210:20:442
444992320,2cyclictest22037-21ssh09:33:322
444992320,2cyclictest15424-21idleruntime-cro07:40:002
444992320,1cyclictest16557-21sh09:29:142
44499230,2cyclictest30007-21rm11:05:302
44499230,23cyclictest26634-21diskmemload10:44:392
44499230,23cyclictest26634-21diskmemload10:11:552
44499230,23cyclictest26634-21diskmemload09:47:122
44499230,23cyclictest0-21swapper/210:32:182
44499230,23cyclictest0-21swapper/210:07:102
44499230,22cyclictest5258-21ssh09:19:192
44499230,22cyclictest24898-21sh11:01:152
44499230,22cyclictest19790-21ssh10:00:112
44499230,22cyclictest0-21swapper/212:25:042
44499230,0cyclictest0-21swapper/210:36:402
44499230,0cyclictest0-21swapper/209:53:202
441992323,0cyclictest0-21swapper/112:35:301
441992323,0cyclictest0-21swapper/111:56:541
441992323,0cyclictest0-21swapper/110:49:111
441992323,0cyclictest0-21swapper/110:12:131
441992323,0cyclictest0-21swapper/109:55:081
441992323,0cyclictest0-21swapper/109:19:121
441992322,1cyclictest3143-21ssh11:38:391
441992321,1cyclictest0-21swapper/112:33:391
441992321,1cyclictest0-21swapper/111:27:201
441992320,2cyclictest12862-21ssh12:15:401
44199230,2cyclictest26657-21apt09:10:011
44199230,23cyclictest26634-21diskmemload09:50:351
44199230,23cyclictest0-21swapper/111:40:351
44199230,23cyclictest0-21swapper/109:33:231
44199230,22cyclictest29419-21ssh12:02:141
44199230,22cyclictest26634-21diskmemload11:07:241
44199230,22cyclictest26634-21diskmemload10:36:111
44199230,22cyclictest22-21ksoftirqd/109:42:201
44199230,22cyclictest19823-21sh10:57:201
44199230,22cyclictest18697-21sh12:20:481
44199230,22cyclictest16152-21ssh10:53:271
44199230,22cyclictest12222-21turbostat12:15:011
44199230,22cyclictest0-21swapper/111:15:211
44199230,0cyclictest0-21swapper/111:14:481
439992323,0cyclictest26634-21diskmemload10:11:460
439992323,0cyclictest0-21swapper/012:31:190
439992323,0cyclictest0-21swapper/012:18:430
439992323,0cyclictest0-21swapper/011:30:340
439992323,0cyclictest0-21swapper/010:51:580
439992322,1cyclictest26634-21diskmemload11:45:240
439992322,1cyclictest26634-21diskmemload11:17:390
439992321,1cyclictest6518-21diskmemload09:20:440
439992321,1cyclictest0-21swapper/011:00:150
439992321,1cyclictest0-21swapper/010:45:090
439992321,1cyclictest0-21swapper/010:31:000
439992321,1cyclictest0-21swapper/009:57:210
439992321,1cyclictest0-21swapper/009:40:520
439992321,1cyclictest0-21swapper/009:30:570
43999230,23cyclictest0-21swapper/010:16:570
43999230,22cyclictest26634-21diskmemload09:50:020
43999230,22cyclictest14867-21ssh10:23:560
43999230,22cyclictest0-21swapper/012:38:460
43999230,22cyclictest0-21swapper/012:20:080
43999230,1cyclictest0-21swapper/010:58:350
43999230,0cyclictest26634-21diskmemload11:08:460
43999230,0cyclictest0-21swapper/011:25:360
43999230,0cyclictest0-21swapper/009:10:360
453992222,0cyclictest18236-21apt-get07:47:183
453992221,1cyclictest7071-21awk08:30:003
453992220,1cyclictest8564-21ssh09:50:213
453992220,1cyclictest20629-21apt-get07:52:163
453992220,1cyclictest0-21swapper/311:05:483
45399220,22cyclictest5813-21ssh11:41:273
45399220,21cyclictest5948-21sh10:45:013
45399220,21cyclictest5327-21ssh10:15:573
45399220,21cyclictest32634-21apt-get08:17:203
45399220,21cyclictest30236-21apt-get08:12:183
45399220,21cyclictest26634-21diskmemload12:35:043
45399220,21cyclictest26634-21diskmemload10:37:273
45399220,21cyclictest26634-21diskmemload09:15:083
45399220,21cyclictest10539-21apt-get07:32:173
45399220,21cyclictest0-21swapper/311:00:583
45399220,21cyclictest0-21swapper/309:40:173
45399220,1cyclictest4013-21apt-get09:47:183
45399220,1cyclictest26634-21diskmemload09:20:523
45399220,1cyclictest24603-21ssh11:29:153
45399220,1cyclictest0-21swapper/307:20:013
45399220,1cyclictest0-21swapper/307:20:013
444992222,0cyclictest0-21swapper/212:20:372
444992222,0cyclictest0-21swapper/211:51:092
444992222,0cyclictest0-21swapper/207:12:192
444992220,2cyclictest30236-21ssh12:31:202
444992220,1cyclictest0-21swapper/211:11:482
44499220,21cyclictest5537-21ssh11:41:062
44499220,21cyclictest26634-21diskmemload12:35:382
44499220,21cyclictest26634-21diskmemload10:16:142
44499220,21cyclictest14258-21ssh11:20:082
44499220,21cyclictest0-21swapper/209:10:162
44499220,21cyclictest0-21swapper/209:10:012
44499220,1cyclictest29480-21sh09:40:282
44499220,1cyclictest26288-21ssh11:31:072
44499220,1cyclictest14171-21ssh09:55:142
44499220,1cyclictest13474-21ssh12:16:252
44499220,0cyclictest0-21swapper/207:47:182
441992222,0cyclictest0-21swapper/110:27:321
441992221,1cyclictest23029-21apt-get07:57:201
441992221,1cyclictest0-21swapper/109:47:041
441992221,1cyclictest0-21swapper/107:27:181
441992220,1cyclictest14264-21perf08:45:001
441992220,1cyclictest0-21swapper/109:05:011
44199220,22cyclictest26634-21diskmemload09:36:131
44199220,21cyclictest26634-21diskmemload10:00:351
44199220,21cyclictest0-21swapper/108:47:191
44199220,1cyclictest1169-21ssh10:40:571
44199220,1cyclictest0-21swapper/111:24:551
439992222,0cyclictest0-21swapper/011:35:300
439992220,1cyclictest5329-21awk07:20:010
439992220,1cyclictest5329-21awk07:20:010
439992220,1cyclictest3575-21ssh11:10:510
439992220,1cyclictest0-21swapper/010:25:330
43999220,1cyclictest16828-21kworker/u8:111:56:490
43999220,1cyclictest11983-21diskmemload09:25:260
45399210,20cyclictest14694-21apt-get08:47:193
45399210,20cyclictest0-21swapper/307:37:183
444992120,0cyclictest0-21swapper/208:17:202
44499210,21cyclictest26634-21diskmemload10:27:232
44499210,0cyclictest0-21swapper/208:35:002
441992119,1cyclictest2241-21cron08:20:011
44199210,20cyclictest0-21swapper/108:12:181
439992121,0cyclictest0-21swapper/007:12:190
439992120,0cyclictest17088-21apt-get08:52:170
439992120,0cyclictest0-21swapper/007:42:180
439992119,1cyclictest2264-21apt08:20:010
439992119,1cyclictest21874-21awk09:00:010
43999210,20cyclictest0-21swapper/007:35:020
43999210,0cyclictest0-21swapper/009:45:300
45399200,20cyclictest0-21swapper/308:37:193
444992020,0cyclictest27843-21apt-get08:07:202
444992019,1cyclictest12302-21apt-get08:42:192
444992018,2cyclictest3332-21apt-get07:17:182
444992018,2cyclictest3332-21apt-get07:17:182
44499200,20cyclictest0-21swapper/208:55:032
441992020,0cyclictest0-21swapper/108:52:181
441992020,0cyclictest0-21swapper/108:32:181
441992020,0cyclictest0-21swapper/107:35:021
441992020,0cyclictest0-21swapper/107:17:181
441992020,0cyclictest0-21swapper/107:17:181
441992019,1cyclictest0-21swapper/108:37:191
441992018,1cyclictest12510-21apt-get07:35:011
44199200,0cyclictest0-21swapper/108:22:191
439992020,0cyclictest0-21swapper/008:47:190
439992020,0cyclictest0-21swapper/007:47:180
439992019,0cyclictest0-21swapper/007:52:160
453991919,0cyclictest0-21swapper/307:57:203
453991919,0cyclictest0-21swapper/307:22:203
453991917,1cyclictest313-21systemd-journal08:05:003
444991919,0cyclictest0-21swapper/207:27:182
444991917,1cyclictest22297-21apt-get09:02:192
444991916,2cyclictest622-21gdbus08:15:012
441991919,0cyclictest0-21swapper/108:02:191
441991919,0cyclictest0-21swapper/107:15:001
441991918,1cyclictest0-21swapper/107:52:161
439991919,0cyclictest0-21swapper/008:42:190
439991919,0cyclictest0-21swapper/008:32:180
439991919,0cyclictest0-21swapper/008:02:190
453991818,0cyclictest0-21swapper/309:02:193
444991818,0cyclictest0-21swapper/208:22:192
441991818,0cyclictest22-21ksoftirqd/108:07:531
441991818,0cyclictest0-21swapper/107:22:201
439991818,0cyclictest0-21swapper/009:07:180
439991818,0cyclictest0-21swapper/008:12:180
439991817,1cyclictest0-21swapper/008:07:200
439991816,1cyclictest11875-21munin-run08:40:000
453991717,0cyclictest0-21swapper/308:52:183
444991715,1cyclictest7715-21basename07:25:012
441991715,1cyclictest15845-21apt-get07:42:181
439991717,0cyclictest0-21swapper/009:02:190
439991717,0cyclictest0-21swapper/007:32:170
439991717,0cyclictest0-21swapper/007:22:200
453991616,0cyclictest0-21swapper/307:12:193
453991614,1cyclictest0-21swapper/307:42:243
45399160,15cyclictest4679-21/usr/sbin/munin08:25:013
444991616,0cyclictest0-21swapper/207:32:172
444991614,1cyclictest0-21swapper/208:47:192
44499160,7cyclictest5096-21apt-get08:27:182
44499160,15cyclictest24500-21ssh07:57:302
439991616,0cyclictest0-21swapper/007:27:180
43999160,0cyclictest0-21swapper/008:27:180
453991515,0cyclictest0-21swapper/308:07:203
444991514,1cyclictest0-21swapper/207:55:002
453991414,0cyclictest0-21swapper/309:05:163
45399140,14cyclictest0-21swapper/308:30:283
45399140,0cyclictest0-21swapper/308:42:163
44499140,14cyclictest0-21swapper/208:03:272
44499140,0cyclictest0-21swapper/208:35:222
44499140,0cyclictest0-21swapper/207:40:332
44199140,14cyclictest0-21swapper/108:59:501
44199140,0cyclictest0-21swapper/107:45:421
43999140,14cyclictest0-21swapper/008:20:040
43999140,14cyclictest0-21swapper/007:57:550
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional