You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-08 - 19:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Sep 08, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2163992524,1cyclictest34-21ksoftirqd/309:15:283
216399250,24cyclictest27891-21diskmemload09:12:423
216399250,24cyclictest27891-21diskmemload09:12:423
2159992523,1cyclictest19824-1kworker/u9:210:46:482
215999250,24cyclictest0-21swapper/211:42:412
215899250,2cyclictest31632-21idleruntime-cro10:40:001
215799250,24cyclictest9071-21rm10:19:230
216399240,23cyclictest0-21swapper/312:27:563
216399240,0cyclictest0-21swapper/309:54:493
215999240,24cyclictest0-21swapper/211:37:222
215999240,23cyclictest27891-21diskmemload12:01:572
215999240,23cyclictest23831-21sh07:55:002
215999240,23cyclictest0-21swapper/211:11:462
215999240,1cyclictest0-21swapper/210:57:432
2158992423,1cyclictest801-21apt-get10:41:491
2158992421,2cyclictest28380-21ssh11:05:561
215899240,24cyclictest0-21swapper/112:32:211
215899240,23cyclictest6862-21ssh11:44:361
215899240,23cyclictest27891-21diskmemload09:50:421
215899240,23cyclictest0-21swapper/111:50:581
215899240,23cyclictest0-21swapper/109:21:471
2157992422,1cyclictest26109-21sh10:05:550
2157992422,1cyclictest20871-21ssh10:01:190
2157992422,1cyclictest20871-21ssh10:01:190
2157992421,2cyclictest0-21swapper/010:48:190
2157992416,4cyclictest19268-21apt-config10:00:010
215799240,23cyclictest640-21sh12:07:260
215799240,23cyclictest23385-21ssh12:28:090
215799240,23cyclictest0-21swapper/012:35:310
215799240,0cyclictest27891-21diskmemload10:43:520
2163992323,0cyclictest27891-21diskmemload11:46:053
2163992323,0cyclictest0-21swapper/312:11:253
2163992323,0cyclictest0-21swapper/311:30:293
2163992323,0cyclictest0-21swapper/309:43:363
2163992322,0cyclictest0-21swapper/310:15:503
2163992321,1cyclictest0-21swapper/311:06:493
216399230,23cyclictest27891-21diskmemload10:08:463
216399230,23cyclictest0-21swapper/312:36:213
216399230,23cyclictest0-21swapper/309:33:023
216399230,22cyclictest8433-21sh11:16:493
216399230,22cyclictest6615-21cstates10:46:483
216399230,22cyclictest6221-21seq09:47:213
216399230,22cyclictest3123-21/usr/sbin/munin11:41:593
216399230,1cyclictest25781-21ssh12:31:113
216399230,0cyclictest0-21swapper/311:12:203
2159992323,0cyclictest27891-21diskmemload12:18:582
2159992323,0cyclictest0-21swapper/210:00:552
2159992323,0cyclictest0-21swapper/210:00:552
2159992323,0cyclictest0-21swapper/209:57:302
2159992321,1cyclictest0-21swapper/211:26:532
215999230,2cyclictest30812-21ssh10:38:542
215999230,23cyclictest27891-21diskmemload09:42:452
215999230,23cyclictest0-21swapper/212:32:122
215999230,23cyclictest0-21swapper/211:51:272
215999230,23cyclictest0-21swapper/211:17:012
215999230,23cyclictest0-21swapper/210:13:162
215999230,23cyclictest0-21swapper/209:37:102
215999230,23cyclictest0-21swapper/209:10:112
215999230,23cyclictest0-21swapper/209:10:112
215999230,22cyclictest10131-21ssh09:51:492
215999230,22cyclictest0-21swapper/212:05:102
215999230,0cyclictest27891-21diskmemload11:32:092
215999230,0cyclictest0-21swapper/210:21:572
2158992323,0cyclictest27891-21diskmemload11:46:471
2158992323,0cyclictest0-21swapper/112:27:591
2158992323,0cyclictest0-21swapper/111:11:421
2158992323,0cyclictest0-21swapper/110:59:221
2158992323,0cyclictest0-21swapper/110:29:561
2158992323,0cyclictest0-21swapper/110:09:271
2158992323,0cyclictest0-21swapper/110:04:491
2158992323,0cyclictest0-21swapper/110:04:491
2158992323,0cyclictest0-21swapper/109:57:471
2158992323,0cyclictest0-21swapper/109:27:441
2158992322,1cyclictest32287-21ssh11:37:431
2158992322,1cyclictest21796-21ssh09:32:151
2158992321,1cyclictest0-21swapper/110:55:011
215899231,21cyclictest21448-21dump-pmu-power07:50:011
215899230,2cyclictest31101-21sh10:10:111
215899230,23cyclictest0-21swapper/112:11:471
215899230,23cyclictest0-21swapper/109:10:291
215899230,23cyclictest0-21swapper/109:10:291
215899230,22cyclictest5470-21ssh10:45:311
215899230,22cyclictest27301-21rm09:37:091
215899230,22cyclictest25983-21grep11:31:551
215899230,22cyclictest22006-21sh10:31:401
215899230,22cyclictest11995-21munin-plugin-st11:20:001
215899230,1cyclictest18474-21sh12:24:001
215899230,0cyclictest27891-21diskmemload11:02:221
215899230,0cyclictest0-21swapper/112:15:421
215899230,0cyclictest0-21swapper/110:20:471
215899230,0cyclictest0-21swapper/109:42:561
2157992323,0cyclictest62550irq/127-eno109:11:050
2157992323,0cyclictest62550irq/127-eno109:11:050
2157992323,0cyclictest0-21swapper/012:10:400
2157992323,0cyclictest0-21swapper/011:46:330
2157992323,0cyclictest0-21swapper/011:07:120
2157992323,0cyclictest0-21swapper/009:16:190
2157992322,1cyclictest62550irq/127-eno110:56:290
2157992321,1cyclictest15399-21rm10:25:110
2157992321,1cyclictest0-21swapper/010:51:020
215799231,21cyclictest62550irq/127-eno111:55:560
215799231,1cyclictest31844-21apt-get08:11:460
215799230,23cyclictest0-21swapper/011:54:250
215799230,23cyclictest0-21swapper/011:28:540
215799230,23cyclictest0-21swapper/010:34:190
215799230,22cyclictest27891-21diskmemload12:02:370
215799230,22cyclictest26621-21ssh11:03:490
215799230,22cyclictest12154-21sh11:20:100
215799230,1cyclictest13717-21sh09:55:000
215799230,1cyclictest0-21swapper/010:11:530
215799230,0cyclictest0-21swapper/012:21:510
215799230,0cyclictest0-21swapper/010:23:130
215799230,0cyclictest0-21swapper/009:46:160
2163992222,0cyclictest1881-21apt-get08:16:473
2163992222,0cyclictest0-21swapper/312:07:463
2163992222,0cyclictest0-21swapper/311:58:513
2163992222,0cyclictest0-21swapper/310:44:083
2163992222,0cyclictest0-21swapper/310:14:383
2163992220,1cyclictest0-21swapper/312:01:073
216399221,0cyclictest0-21swapper/310:25:373
216399220,21cyclictest29448-21apt-get08:06:463
216399220,21cyclictest27891-21diskmemload10:21:113
216399220,21cyclictest13511-21idleruntime-cro08:40:013
216399220,21cyclictest0-21swapper/311:51:263
216399220,21cyclictest0-21swapper/311:35:033
216399220,21cyclictest0-21swapper/311:25:113
216399220,21cyclictest0-21swapper/311:20:023
216399220,21cyclictest0-21swapper/311:00:323
216399220,21cyclictest0-21swapper/310:56:103
216399220,21cyclictest0-21swapper/309:57:303
216399220,21cyclictest0-21swapper/309:36:233
216399220,21cyclictest0-21swapper/307:26:473
216399220,1cyclictest0-21swapper/310:52:083
216399220,1cyclictest0-21swapper/310:35:483
216399220,1cyclictest0-21swapper/310:30:133
216399220,0cyclictest27891-21diskmemload12:22:053
216399220,0cyclictest0-21swapper/312:17:273
2159992222,0cyclictest0-21swapper/212:37:252
2159992222,0cyclictest0-21swapper/210:25:022
2159992222,0cyclictest0-21swapper/210:08:562
2159992222,0cyclictest0-21swapper/209:19:012
2159992220,1cyclictest3761-21sh12:11:242
2159992220,1cyclictest31994-21ssh10:40:122
2159992220,1cyclictest18583-21ssh09:30:052
2159992220,1cyclictest0-21swapper/208:00:002
215999220,22cyclictest27891-21diskmemload09:47:252
215999220,21cyclictest7789-21ssh09:20:282
215999220,21cyclictest22562-21sh12:26:572
215999220,21cyclictest22539-21ssh11:00:352
215999220,21cyclictest0-21swapper/211:56:002
215999220,21cyclictest0-21swapper/211:05:262
215999220,21cyclictest0-21swapper/210:30:142
215999220,21cyclictest0-21swapper/207:31:452
215999220,1cyclictest0-21swapper/210:54:462
215999220,0cyclictest0-21swapper/212:20:022
215999220,0cyclictest0-21swapper/211:46:132
2158992222,0cyclictest0-21swapper/111:22:241
2158992222,0cyclictest0-21swapper/110:17:501
2158992220,1cyclictest28653-21sh08:05:001
2158992220,1cyclictest20717-21munin-run08:55:001
215899220,22cyclictest27891-21diskmemload09:19:521
215899220,21cyclictest783-21snmpd12:37:191
215899220,21cyclictest24032-21ssh12:00:031
215899220,21cyclictest18919-21rm11:55:401
215899220,21cyclictest0-21swapper/108:06:461
215899220,1cyclictest6961-21sendmail-msp07:20:001
215899220,1cyclictest0-21swapper/111:28:591
215899220,0cyclictest0-21swapper/109:47:261
2157992222,0cyclictest0-21swapper/012:30:020
2157992221,1cyclictest0-21swapper/011:40:150
2157992220,1cyclictest11080-21idleruntime-cro08:35:010
2157992220,1cyclictest0-21swapper/011:14:130
2157992219,2cyclictest1365-21sh09:43:240
2157992219,2cyclictest12390-21ssh12:18:220
215799220,21cyclictest0-21swapper/011:32:070
215799220,21cyclictest0-21swapper/011:16:480
215799220,21cyclictest0-21swapper/008:56:450
215799220,21cyclictest0-21swapper/007:46:470
215799220,21cyclictest0-21swapper/007:31:450
215799220,1cyclictest26759-21ssh10:35:400
215799220,1cyclictest18628-21ssh09:30:090
215799220,1cyclictest11168-21ssh09:22:530
215799220,1cyclictest0-21swapper/009:35:430
2163992121,0cyclictest0-21swapper/307:46:463
2163992121,0cyclictest0-21swapper/307:41:453
2163992120,0cyclictest0-21swapper/307:16:463
216399210,20cyclictest0-21swapper/308:21:453
216399210,0cyclictest27891-21diskmemload09:24:043
2159992119,1cyclictest6322-21awk08:25:012
215999210,21cyclictest7999-21kworker/u8:009:26:362
215999210,20cyclictest26661-21apt-get08:01:472
215999210,20cyclictest19453-21apt-get07:46:472
215999210,20cyclictest0-21swapper/208:11:462
215999210,0cyclictest0-21swapper/208:16:462
2158992121,0cyclictest0-21swapper/107:26:471
2158992119,1cyclictest23097-21apt09:00:001
215899210,20cyclictest14647-21apt-get07:36:491
215899210,20cyclictest0-21swapper/108:16:471
215899210,0cyclictest0-21swapper/108:41:471
2157992121,0cyclictest0-21swapper/007:41:450
2157992120,1cyclictest0-21swapper/009:25:090
2157992119,1cyclictest0-21swapper/007:30:000
2157992118,2cyclictest27892-21idleruntime-cro09:10:000
2163992020,0cyclictest0-21swapper/308:11:463
2163992020,0cyclictest0-21swapper/307:51:463
2163992020,0cyclictest0-21swapper/307:36:503
2163992020,0cyclictest0-21swapper/307:21:473
216399200,0cyclictest21095-21apt-get08:56:453
2159992020,0cyclictest0-21swapper/208:56:452
2159992018,1cyclictest4514-21cron07:15:002
215999200,0cyclictest0-21swapper/208:06:462
2158992020,0cyclictest23511-21apt-get09:01:441
2158992020,0cyclictest0-21swapper/108:46:491
2157992018,1cyclictest31445-21grep08:10:000
215799200,19cyclictest0-21swapper/009:01:450
215799200,0cyclictest0-21swapper/008:41:470
215799200,0cyclictest0-21swapper/008:16:470
2163991919,0cyclictest0-21swapper/308:41:473
2163991917,1cyclictest18300-21awk08:50:003
2163991917,1cyclictest0-21swapper/308:35:013
2158991919,0cyclictest0-21swapper/108:36:451
2157991917,1cyclictest0-21swapper/007:55:000
2163991818,0cyclictest0-21swapper/309:06:463
216399180,17cyclictest0-21swapper/307:56:453
2158991818,0cyclictest17054-21apt-get07:41:461
2157991818,0cyclictest0-21swapper/008:51:460
2157991818,0cyclictest0-21swapper/008:26:480
2157991816,1cyclictest11494-21apt-get08:36:450
2163991717,0cyclictest12141-21apt-get07:31:453
2163991717,0cyclictest0-21swapper/308:51:463
2163991717,0cyclictest0-21swapper/307:11:463
2159991715,2cyclictest25906-21apt-get09:06:462
2158991716,0cyclictest0-21swapper/108:26:481
2159991616,0cyclictest0-21swapper/208:46:492
2159991616,0cyclictest0-21swapper/207:41:452
2159991615,1cyclictest6708-21apt-get08:26:482
2159991615,1cyclictest18699-21apt-get08:51:462
2159991615,1cyclictest13913-21apt-get08:41:472
2159991614,1cyclictest0-21swapper/207:36:592
215999160,1cyclictest313-21systemd-journal08:40:012
215899160,1cyclictest1673-21latency_hist08:15:011
2157991614,1cyclictest24881-21head07:56:470
2163991512,2cyclictest24253-21if_enp1s0.809:01:473
215999150,14cyclictest0-21swapper/207:16:482
2158991515,0cyclictest0-21swapper/107:51:461
215799150,9cyclictest313-21systemd-journal07:20:000
216399140,14cyclictest0-21swapper/308:26:443
216399140,14cyclictest0-21swapper/308:00:163
215999140,6cyclictest0-21swapper/209:01:442
215999140,14cyclictest0-21swapper/208:32:252
215999140,14cyclictest0-21swapper/207:25:142
215999140,14cyclictest0-21swapper/207:20:562
2158991414,0cyclictest0-21swapper/107:56:441
2158991414,0cyclictest0-21swapper/107:11:461
2158991413,1cyclictest22-21ksoftirqd/109:05:431
215899140,0cyclictest0-21swapper/108:20:471
215899140,0cyclictest0-21swapper/107:20:221
2157991414,0cyclictest0-21swapper/008:20:070
2157991414,0cyclictest0-21swapper/007:35:480
215799140,0cyclictest0-21swapper/008:45:080
215799140,0cyclictest0-21swapper/008:00:230
215799140,0cyclictest0-21swapper/007:20:370
215799140,0cyclictest0-21swapper/007:11:420
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional