You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-21 - 17:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Thu May 21, 2026 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17315992823,4cyclictest0-21swapper/212:23:122
1732099260,25cyclictest6721-21sh09:34:373
17315992620,2cyclictest542-21ssh10:54:052
1731599260,1cyclictest31093-21ssh10:23:462
1731599260,1cyclictest0-21swapper/210:34:022
17311992624,2cyclictest11352-21sh10:35:191
1731199260,26cyclictest0-21swapper/109:50:481
1730999260,25cyclictest11047-21diskmemload10:27:190
17320992523,1cyclictest11047-21diskmemload12:32:033
17315992522,0cyclictest0-21swapper/209:30:462
17315992521,0cyclictest11047-21diskmemload09:36:332
17315992521,0cyclictest0-21swapper/209:20:022
17315992521,0cyclictest0-21swapper/209:20:022
17315992520,4cyclictest0-21swapper/211:39:352
17315992519,2cyclictest1764-21rm12:19:462
17315992518,2cyclictest24803-21sendmail12:40:012
1731599250,24cyclictest27908-21sh12:14:002
1731599250,24cyclictest0-21swapper/212:01:512
1731599250,21cyclictest0-21swapper/212:32:302
1731599250,1cyclictest0-21swapper/211:21:342
1731599250,1cyclictest0-21swapper/209:14:012
1731199250,24cyclictest0-21swapper/111:27:031
1730999250,24cyclictest11047-21diskmemload09:12:090
1730999250,1cyclictest0-21swapper/009:16:300
17320992422,1cyclictest0-21swapper/311:51:293
1732099240,23cyclictest27417-21kworker/u8:311:48:353
1732099240,23cyclictest11774-21apt-get11:32:433
1732099240,23cyclictest11047-21diskmemload12:21:133
1732099240,23cyclictest11047-21diskmemload11:00:553
1732099240,23cyclictest0-21swapper/309:14:023
17315992424,0cyclictest0-21swapper/211:32:262
1731599240,23cyclictest0-21swapper/211:08:582
1731599240,23cyclictest0-21swapper/210:58:472
1731599240,20cyclictest0-21swapper/212:27:382
1731599240,1cyclictest0-21swapper/211:25:492
17311992424,0cyclictest11047-21diskmemload10:42:191
17311992423,1cyclictest28050-21ssh12:14:101
17311992423,0cyclictest0-21swapper/110:58:021
17311992421,2cyclictest23903-21ssh12:38:491
1731199241,1cyclictest151rcu_preempt11:37:251
1731199240,24cyclictest0-21swapper/110:10:151
1731199240,24cyclictest0-21swapper/109:58:501
1731199240,23cyclictest0-21swapper/112:28:171
1731199240,23cyclictest0-21swapper/111:33:411
1731199240,23cyclictest0-21swapper/109:33:211
17309992424,0cyclictest11047-21diskmemload09:22:560
17309992424,0cyclictest11047-21diskmemload09:22:560
17309992423,1cyclictest23264-21ssh09:48:480
17309992423,1cyclictest13-21ksoftirqd/009:50:480
1730999240,24cyclictest11047-21diskmemload11:57:040
1730999240,24cyclictest0-21swapper/012:07:550
1730999240,24cyclictest0-21swapper/011:38:200
1730999240,23cyclictest6283-21ssh10:59:070
1730999240,23cyclictest11047-21diskmemload12:33:330
1730999240,23cyclictest11047-21diskmemload10:03:530
1730999240,0cyclictest0-21swapper/009:37:390
17320992323,0cyclictest11047-21diskmemload10:55:303
17320992323,0cyclictest0-21swapper/312:28:583
17320992323,0cyclictest0-21swapper/311:40:153
17320992323,0cyclictest0-21swapper/310:06:253
17320992323,0cyclictest0-21swapper/309:38:133
17320992321,1cyclictest19020-21perf08:20:003
17320992321,1cyclictest11047-21diskmemload09:26:443
17320992321,1cyclictest0-21swapper/311:16:513
17320992321,1cyclictest0-21swapper/309:41:523
17320992320,2cyclictest31442-21latency_hist08:45:013
17320992320,2cyclictest23919-21ssh09:21:523
17320992320,2cyclictest23919-21ssh09:21:523
17320992320,2cyclictest2085-21sh11:22:523
1732099230,2cyclictest11047-21diskmemload10:41:263
1732099230,23cyclictest11047-21diskmemload11:36:363
1732099230,23cyclictest0-21swapper/310:10:053
1732099230,22cyclictest29105-21taskset10:51:193
1732099230,22cyclictest27177-21rm10:20:563
1732099230,22cyclictest22737-21ssh09:48:083
1732099230,22cyclictest14470-21fw_packets12:02:443
1732099230,22cyclictest0-21swapper/312:15:013
1732099230,22cyclictest0-21swapper/311:25:033
1732099230,22cyclictest0-21swapper/311:09:533
1732099230,1cyclictest3090-21perf10:00:003
17315992320,2cyclictest6685-21sh11:55:432
1731599230,2cyclictest0-21swapper/212:06:462
1731599230,23cyclictest0-21swapper/210:39:212
1731599230,23cyclictest0-21swapper/210:08:512
1731599230,23cyclictest0-21swapper/210:00:482
1731599230,22cyclictest11047-21diskmemload09:49:222
1731599230,0cyclictest27318-21ssh11:45:072
1731599230,0cyclictest0-21swapper/210:26:562
1731599230,0cyclictest0-21swapper/209:56:242
1731599230,0cyclictest0-21swapper/209:41:472
17311992323,0cyclictest11047-21diskmemload11:41:361
17311992323,0cyclictest0-21swapper/112:17:321
17311992323,0cyclictest0-21swapper/111:45:321
17311992323,0cyclictest0-21swapper/110:05:551
17311992321,1cyclictest7150-21ssh11:56:141
17311992321,1cyclictest22043-21sh07:20:001
17311992321,1cyclictest14782-21ssh11:06:591
17311992321,1cyclictest0-21swapper/111:22:281
17311992321,1cyclictest0-21swapper/109:11:171
17311992320,2cyclictest6140-21ssh12:22:591
1731199230,22cyclictest8311-21sh11:00:541
1731199230,22cyclictest4674-21ssh10:01:511
1731199230,22cyclictest28340-21ssh10:50:131
1731199230,22cyclictest18101-21ssh12:05:451
1731199230,22cyclictest11068-21turbostat09:10:001
1731199230,22cyclictest11047-21diskmemload11:17:401
1731199230,22cyclictest10086-21/usr/sbin/munin09:37:441
1731199230,22cyclictest0-21swapper/111:54:011
1731199230,22cyclictest0-21swapper/110:32:451
1731199230,1cyclictest21799-21ssh10:16:191
1731199230,1cyclictest0-21swapper/109:41:161
1731199230,0cyclictest0-21swapper/110:46:271
1731199230,0cyclictest0-21swapper/109:18:131
17309992323,0cyclictest11047-21diskmemload11:03:580
17309992323,0cyclictest0-21swapper/011:42:070
17309992323,0cyclictest0-21swapper/010:20:240
17309992323,0cyclictest0-21swapper/010:12:260
17309992322,1cyclictest0-21swapper/010:30:380
1730999232,1cyclictest62550irq/127-eno111:31:180
17309992321,2cyclictest17263-21ssh09:43:230
17309992321,1cyclictest8324-21ssh12:25:270
17309992321,1cyclictest24431-21ssh11:14:540
17309992321,1cyclictest2346-21ssh12:20:100
17309992321,1cyclictest16592-21cron08:15:000
1730999230,23cyclictest11479-21ssh10:35:290
1730999230,23cyclictest0-21swapper/012:11:040
1730999230,23cyclictest0-21swapper/011:54:270
1730999230,23cyclictest0-21swapper/011:29:120
1730999230,23cyclictest0-21swapper/010:42:020
1730999230,23cyclictest0-21swapper/009:30:570
1730999230,23cyclictest0-21swapper/009:27:300
1730999230,22cyclictest31905-21ssh11:48:570
1730999230,22cyclictest31369-21ssh11:20:420
1730999230,22cyclictest28384-21ssh10:50:160
1730999230,22cyclictest27961-21perf10:50:000
1730999230,22cyclictest26346-21ssh11:16:410
1730999230,22cyclictest12895-21sh12:01:200
1730999230,22cyclictest0-21swapper/011:07:160
1730999230,0cyclictest0-21swapper/012:36:340
1730999230,0cyclictest0-21swapper/012:15:340
17320992222,0cyclictest11047-21diskmemload10:35:093
17320992222,0cyclictest0-21swapper/312:11:143
17320992222,0cyclictest0-21swapper/309:19:243
17320992222,0cyclictest0-21swapper/308:07:413
17320992221,1cyclictest0-21swapper/309:51:443
17320992220,1cyclictest0-21swapper/312:06:143
1732099220,22cyclictest26472-21ssh10:48:043
1732099220,22cyclictest20332-21ssh11:11:403
1732099220,22cyclictest11047-21diskmemload10:16:553
1732099220,21cyclictest11047-21diskmemload12:35:123
1732099220,21cyclictest11047-21diskmemload11:56:273
1732099220,21cyclictest0-21swapper/310:25:173
1732099220,21cyclictest0-21swapper/308:05:013
1732099220,1cyclictest29831-21apt-get07:37:413
1732099220,1cyclictest0-21swapper/310:01:043
1732099220,0cyclictest0-21swapper/310:30:053
1732099220,0cyclictest0-21swapper/308:32:413
17315992222,0cyclictest0-21swapper/210:49:162
17315992222,0cyclictest0-21swapper/209:15:032
17315992221,1cyclictest0-21swapper/211:53:352
17315992220,1cyclictest3813-21perf08:55:002
17315992220,1cyclictest0-21swapper/211:17:532
17315992220,1cyclictest0-21swapper/211:10:052
1731599220,22cyclictest0-21swapper/209:25:052
1731599220,21cyclictest9016-21latency_hist08:00:012
1731599220,21cyclictest20267-21apt-get10:15:022
1731599220,21cyclictest0-21swapper/211:01:022
1731599220,1cyclictest0-21swapper/208:35:012
1731599220,0cyclictest0-21swapper/210:44:052
17311992222,0cyclictest0-21swapper/112:31:501
17311992222,0cyclictest0-21swapper/110:20:091
17311992222,0cyclictest0-21swapper/109:46:171
17311992220,1cyclictest23803-21ssh09:21:431
17311992220,1cyclictest23803-21ssh09:21:431
1731199220,21cyclictest3868-21sh10:28:121
1731199220,21cyclictest28605-21sh09:25:301
1731199220,21cyclictest0-21swapper/112:00:581
1731199220,21cyclictest0-21swapper/111:13:351
1731199220,1cyclictest6233-21munin-run09:00:001
1731199220,1cyclictest0-21swapper/107:50:011
1730999220,22cyclictest0-21swapper/010:05:400
1730999220,21cyclictest21046-21ssh10:15:130
1730999220,21cyclictest0-21swapper/007:52:420
1730999220,1cyclictest31892-21apt-get08:47:420
1730999220,1cyclictest0-21swapper/009:57:320
17320992121,0cyclictest0-21swapper/309:02:423
17320992121,0cyclictest0-21swapper/307:12:423
17320992119,1cyclictest9136-21apt-get09:07:423
17320992119,1cyclictest2228-21apt-get07:47:413
17320992119,1cyclictest14678-21apt-get08:12:433
17315992121,0cyclictest0-21swapper/207:12:422
17315992119,1cyclictest313-21systemd-journal09:05:002
1731599210,21cyclictest0-21swapper/208:07:412
1731599210,20cyclictest27331-21apt-get07:32:392
1731599210,20cyclictest0-21swapper/208:12:432
1731599210,1cyclictest0-21swapper/208:42:432
1731599210,0cyclictest31892-21apt-get08:47:422
17311992118,2cyclictest8677-21latency_hist09:05:001
1731199210,20cyclictest0-21swapper/108:42:431
1731199210,20cyclictest0-21swapper/107:32:391
17309992121,0cyclictest0-21swapper/008:22:410
17309992121,0cyclictest0-21swapper/007:22:410
17309992119,1cyclictest6234-21grep09:00:000
1730999210,20cyclictest29498-21apt-get08:42:430
1730999210,20cyclictest0-21swapper/008:00:020
1730999210,0cyclictest0-21swapper/008:52:400
1730999210,0cyclictest0-21swapper/007:47:410
17320992020,0cyclictest24936-21apt-get07:27:413
17320992018,1cyclictest21891-21apt-get08:27:413
17315992019,1cyclictest32250-21apt-get07:42:422
1731599200,0cyclictest17071-21apt-get08:17:412
17311992020,0cyclictest9886-21apt-get08:02:411
17311992020,0cyclictest0-21swapper/108:17:401
17311992018,1cyclictest26684-21apt-get08:37:411
1731199200,20cyclictest0-21swapper/108:22:421
17309992020,0cyclictest0-21swapper/007:57:410
17309992018,1cyclictest11065-21apt09:10:000
1730999200,20cyclictest0-21swapper/008:37:410
17320991919,0cyclictest0-21swapper/308:57:413
17320991919,0cyclictest0-21swapper/307:17:413
17320991917,1cyclictest3821-21munin-run08:55:003
17315991919,0cyclictest0-21swapper/207:37:412
17315991917,1cyclictest6223-21grep09:00:002
17311991919,0cyclictest17728-21apt-get07:12:421
17311991919,0cyclictest0-21swapper/107:57:411
17311991919,0cyclictest0-21swapper/107:37:411
17311991919,0cyclictest0-21swapper/107:27:411
17309991918,1cyclictest27331-21apt-get07:32:390
1730999190,18cyclictest19013-21grep08:20:000
17320991818,0cyclictest0-21swapper/307:57:413
1731599182,15cyclictest3390-21sed07:47:472
17315991818,0cyclictest0-21swapper/208:02:422
17315991816,1cyclictest24457-21awk07:25:002
17311991817,0cyclictest0-21swapper/108:52:411
17309991816,1cyclictest14207-21turbostat.cron08:10:000
17320991717,0cyclictest0-21swapper/307:32:393
1732099170,15cyclictest0-21swapper/307:22:413
17315991717,0cyclictest19498-21apt-get08:22:412
17315991716,0cyclictest20123-21apt-get07:17:412
17315991715,1cyclictest28624-21apt-get08:40:012
17311991716,0cyclictest12283-21apt-get08:07:411
17320991615,0cyclictest4626-21apt-get07:52:423
17320991614,1cyclictest0-21swapper/307:42:443
1731199163,2cyclictest151rcu_preempt08:12:451
17311991616,0cyclictest0-21swapper/108:47:421
17311991616,0cyclictest0-21swapper/108:27:411
17309991614,2cyclictest13-21ksoftirqd/007:27:410
17309991614,1cyclictest0-21swapper/009:05:000
17315991515,0cyclictest0-21swapper/208:27:412
17315991515,0cyclictest0-21swapper/207:52:422
17315991514,0cyclictest0-21swapper/207:27:412
17320991414,0cyclictest0-21swapper/308:45:103
1732099140,14cyclictest0-21swapper/308:35:023
1732099140,0cyclictest0-21swapper/308:21:063
1731599140,14cyclictest0-21swapper/209:06:052
17311991414,0cyclictest0-21swapper/108:30:211
1731199140,0cyclictest0-21swapper/107:50:031
17309991414,0cyclictest0-21swapper/008:30:030
17309991414,0cyclictest0-21swapper/007:41:060
1730999140,14cyclictest0-21swapper/008:25:080
1730999140,14cyclictest0-21swapper/007:35:120
1730999140,14cyclictest0-21swapper/007:11:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional