You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-02 - 13:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Jun 02, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1791993330,1cyclictest13-21ksoftirqd/022:59:360
1791992827,1cyclictest13-21ksoftirqd/023:35:160
179199260,2cyclictest31070-21expr21:43:410
180299250,24cyclictest27826-21diskmemload23:07:583
180299250,24cyclictest0-21swapper/321:14:043
1796992523,1cyclictest0-21swapper/200:08:442
1796992519,2cyclictest17536-21ssh23:31:382
1791992523,0cyclictest27826-21diskmemload23:18:130
180299242,21cyclictest326-21sh00:15:573
1802992422,1cyclictest11408-21sh21:54:403
1802992422,1cyclictest0-21swapper/322:58:393
180299241,22cyclictest783-21snmpd23:18:523
180299240,23cyclictest27826-21diskmemload21:20:353
180299240,23cyclictest0-21swapper/323:21:113
180299240,1cyclictest17676-21ssh23:00:593
180299240,0cyclictest0-21swapper/322:18:403
1796992423,1cyclictest0-21swapper/221:37:392
1796992422,1cyclictest606-21dbus-daemon19:25:002
1796992422,1cyclictest0-21swapper/222:40:402
1796992421,3cyclictest0-21swapper/223:16:432
1796992420,3cyclictest30980-21awk20:10:012
179699240,23cyclictest27826-21diskmemload23:55:232
179699240,23cyclictest27826-21diskmemload21:55:102
179699240,23cyclictest0-21swapper/200:38:132
1793992424,0cyclictest0-21swapper/122:39:131
1793992424,0cyclictest0-21swapper/100:03:061
1793992422,1cyclictest0-21swapper/123:35:001
179399240,24cyclictest0-21swapper/123:46:281
179399240,24cyclictest0-21swapper/121:18:261
1791992423,1cyclictest13-21ksoftirqd/019:40:000
1791992423,1cyclictest0-21swapper/023:08:440
1791992423,1cyclictest0-21swapper/021:52:410
1791992421,2cyclictest30975-21sh21:13:440
179199241,22cyclictest62550irq/127-eno122:47:010
179199241,22cyclictest0-21swapper/021:37:360
179199240,23cyclictest6584-21ssh23:21:100
179199240,23cyclictest5729-21ssh21:49:030
179199240,1cyclictest3035-21ssh22:17:550
179199240,1cyclictest18235-21ssh23:32:490
179199240,1cyclictest14100-21ssh21:28:090
179199240,1cyclictest14100-21ssh21:28:090
1802992323,0cyclictest0-21swapper/323:12:013
1802992323,0cyclictest0-21swapper/300:05:123
1802992322,1cyclictest11582-21ssh23:55:533
1802992321,1cyclictest313-21systemd-journal20:00:013
1802992321,1cyclictest0-21swapper/300:12:073
1802992320,2cyclictest28897-21ssh22:41:543
1802992320,2cyclictest15359-21sh00:29:563
180299230,2cyclictest4987-21seq22:49:023
180299230,2cyclictest27593-21ssh21:39:203
180299230,23cyclictest0-21swapper/323:43:163
180299230,23cyclictest0-21swapper/323:33:553
180299230,23cyclictest0-21swapper/321:43:333
180299230,23cyclictest0-21swapper/300:22:193
180299230,22cyclictest0-21swapper/321:32:483
180299230,0cyclictest27826-21diskmemload21:18:193
180299230,0cyclictest0-21swapper/300:34:483
1796992323,0cyclictest1783-21cyclictest22:59:082
1796992323,0cyclictest0-21swapper/221:16:052
1796992323,0cyclictest0-21swapper/200:24:462
1796992321,1cyclictest669-21sh21:14:572
1796992321,1cyclictest0-21swapper/221:27:282
1796992321,1cyclictest0-21swapper/221:27:282
1796992321,1cyclictest0-21swapper/200:15:072
1796992320,2cyclictest28733-21ssh22:10:572
179699230,2cyclictest20030-21fschecks_time22:33:422
179699230,2cyclictest11389-21cron22:55:002
179699230,2cyclictest0-21swapper/200:26:392
179699230,23cyclictest0-21swapper/222:07:292
179699230,23cyclictest0-21swapper/200:34:402
179699230,22cyclictest7881-21ssh23:53:192
179699230,22cyclictest7741-21sh21:51:412
179699230,22cyclictest5945-21cut22:50:012
179699230,22cyclictest4128-21df21:48:402
179699230,22cyclictest27826-21diskmemload22:20:272
179699230,22cyclictest0-21swapper/223:42:442
179699230,22cyclictest0-21swapper/223:00:352
179699230,1cyclictest27826-21diskmemload21:20:242
179699230,0cyclictest0-21swapper/223:36:212
179699230,0cyclictest0-21swapper/223:09:272
179699230,0cyclictest0-21swapper/221:32:422
179699230,0cyclictest0-21swapper/220:43:402
1793992323,0cyclictest27826-21diskmemload23:43:031
1793992323,0cyclictest27826-21diskmemload22:05:271
1793992323,0cyclictest0-21swapper/123:17:071
1793992323,0cyclictest0-21swapper/123:08:371
1793992323,0cyclictest0-21swapper/122:58:171
1793992323,0cyclictest0-21swapper/122:50:121
1793992323,0cyclictest0-21swapper/121:37:391
1793992323,0cyclictest0-21swapper/100:30:451
1793992323,0cyclictest0-21swapper/100:26:401
1793992323,0cyclictest0-21swapper/100:16:321
1793992322,1cyclictest21590-21ssh22:04:031
1793992321,1cyclictest6124-21sh23:50:351
1793992321,1cyclictest22531-21ssh00:37:201
1793992321,1cyclictest0-21swapper/123:00:471
1793992320,2cyclictest17434-21rm22:30:351
1793992320,2cyclictest11146-21sh23:55:141
179399230,2cyclictest28297-21sh00:12:071
179399230,2cyclictest11494-21taskset21:24:311
179399230,23cyclictest0-21swapper/123:36:541
179399230,23cyclictest0-21swapper/122:40:531
179399230,23cyclictest0-21swapper/122:15:471
179399230,23cyclictest0-21swapper/121:40:421
179399230,23cyclictest0-21swapper/121:11:531
179399230,22cyclictest8272-21apt-get23:23:401
179399230,22cyclictest5754-21ssh00:21:041
179399230,22cyclictest28047-21sh23:10:431
179399230,22cyclictest27826-21diskmemload22:21:411
179399230,1cyclictest27826-21diskmemload21:34:581
179399230,0cyclictest27826-21diskmemload22:10:191
179399230,0cyclictest0-21swapper/121:52:341
179399230,0cyclictest0-21swapper/121:45:281
1791992323,0cyclictest27826-21diskmemload23:56:170
1791992323,0cyclictest27826-21diskmemload22:36:200
1791992322,1cyclictest62550irq/127-eno123:46:510
1791992322,1cyclictest62550irq/127-eno121:24:020
1791992321,1cyclictest0-21swapper/023:25:240
1791992321,1cyclictest0-21swapper/021:31:370
1791992321,1cyclictest0-21swapper/000:31:170
1791992321,1cyclictest0-21swapper/000:05:140
1791992320,2cyclictest6208-21rm23:50:420
1791992320,2cyclictest27056-21ssh00:10:120
179199230,2cyclictest12579-21ssh00:28:210
179199230,23cyclictest0-21swapper/022:44:100
179199230,23cyclictest0-21swapper/022:33:490
179199230,23cyclictest0-21swapper/022:27:060
179199230,23cyclictest0-21swapper/022:24:060
179199230,23cyclictest0-21swapper/000:19:140
179199230,22cyclictest313-21systemd-journal20:25:000
179199230,22cyclictest24954-21ssh22:08:110
179199230,22cyclictest17233-21ssh00:01:240
179199230,22cyclictest10023-21sh00:24:510
179199230,22cyclictest0-21swapper/023:02:280
179199230,22cyclictest0-21swapper/022:53:560
179199230,22cyclictest0-21swapper/021:58:420
179199230,22cyclictest0-21swapper/019:58:390
179199230,0cyclictest0-21swapper/000:37:100
1802992222,0cyclictest0-21swapper/323:50:273
1802992222,0cyclictest0-21swapper/322:35:253
1802992221,1cyclictest11185-21apt-get20:38:403
1802992221,1cyclictest0-21swapper/322:25:263
1802992220,1cyclictest12918-21sh21:56:343
1802992220,1cyclictest0-21swapper/321:10:003
1802992219,2cyclictest1119-21rm23:46:113
180299220,21cyclictest27826-21diskmemload22:20:343
180299220,21cyclictest27826-21diskmemload22:10:283
180299220,21cyclictest27826-21diskmemload21:30:013
180299220,21cyclictest27826-21diskmemload21:30:013
180299220,21cyclictest22081-21ssh23:35:313
180299220,21cyclictest21928-21apt-get19:53:393
180299220,21cyclictest0-21swapper/300:36:203
180299220,1cyclictest9405-21cut19:25:003
180299220,1cyclictest3024-21sh21:47:363
180299220,1cyclictest18195-21ssh22:01:353
180299220,1cyclictest17557-21ssh00:01:523
180299220,0cyclictest0-21swapper/322:06:023
1796992222,0cyclictest0-21swapper/223:48:032
1796992222,0cyclictest0-21swapper/222:04:112
1796992221,1cyclictest0-21swapper/223:11:242
1796992221,1cyclictest0-21swapper/200:00:562
1796992220,1cyclictest25044-21apt-get21:05:022
1796992220,1cyclictest24322-21apt-get19:58:392
1796992220,1cyclictest22702-21ssh22:35:362
1796992220,1cyclictest0-21swapper/222:26:202
1796992220,1cyclictest0-21swapper/200:10:542
179699220,21cyclictest16600-21munin-plugin-st19:40:002
179699220,21cyclictest0-21swapper/223:21:352
179699220,21cyclictest0-21swapper/222:15:552
179699220,1cyclictest974-21munin-run20:15:002
1793992219,2cyclictest27826-21diskmemload22:45:381
179399220,21cyclictest9926-21apt-get19:28:381
179399220,21cyclictest7532-21apt-get19:23:371
179399220,21cyclictest3988-21apt-get20:23:361
179399220,21cyclictest29132-21apt-get20:08:381
179399220,21cyclictest13810-21ssh21:57:571
179399220,21cyclictest0-21swapper/120:03:381
179399220,1cyclictest16604-21perf19:40:001
179399220,1cyclictest13863-21ssh21:27:491
179399220,1cyclictest13863-21ssh21:27:491
179399220,1cyclictest11520-21sh23:25:351
1791992222,0cyclictest0-21swapper/022:12:430
1791992220,1cyclictest25051-21latency_hist21:05:000
179199220,22cyclictest0-21swapper/022:02:430
179199220,22cyclictest0-21swapper/021:18:530
179199220,21cyclictest27749-21ssh23:40:270
179199220,21cyclictest0-21swapper/019:53:390
179199220,1cyclictest21384-21latency_hist19:50:000
179199220,0cyclictest0-21swapper/023:13:410
1802992121,0cyclictest0-21swapper/319:18:383
1802992120,1cyclictest0-21swapper/323:25:143
1802992120,0cyclictest26737-21apt-get20:03:383
1802992118,2cyclictest3441-21latency_hist20:20:003
180299210,21cyclictest0-21swapper/321:03:403
180299210,20cyclictest0-21swapper/320:13:393
180299210,1cyclictest6340-21ssh22:50:023
1796992120,1cyclictest18394-21apt-get20:53:422
1796992119,1cyclictest0-21swapper/219:43:402
179399210,20cyclictest0-21swapper/120:58:391
179399210,20cyclictest0-21swapper/120:18:391
1791992120,1cyclictest0-21swapper/021:05:020
1791992119,1cyclictest8246-21grep20:30:000
179199210,20cyclictest0-21swapper/019:28:380
1802992020,0cyclictest0-21swapper/319:43:403
1802992020,0cyclictest0-21swapper/319:33:393
1802992018,1cyclictest9926-21apt-get19:28:383
180299200,1cyclictest619-21polkitd20:35:003
1796992018,1cyclictest23203-21apt-get21:03:402
179699200,0cyclictest0-21swapper/219:13:372
1793992020,0cyclictest0-21swapper/121:05:021
1793992020,0cyclictest0-21swapper/120:28:401
1793992020,0cyclictest0-21swapper/119:43:401
1791992020,0cyclictest0-21swapper/019:33:390
179199200,20cyclictest2035-21kworker/0:019:12:160
1802991918,1cyclictest13602-21apt-get20:43:403
1802991917,1cyclictest22673-21perf21:00:013
1796991919,0cyclictest0-21swapper/219:33:392
1793991919,0cyclictest0-21swapper/121:03:411
1793991919,0cyclictest0-21swapper/119:53:391
1793991917,1cyclictest0-21swapper/120:40:011
179199195,14cyclictest12142-21kworker/0:120:41:520
1791991919,0cyclictest20784-21apt-get20:58:380
1791991919,0cyclictest0-21swapper/020:53:420
179199191,18cyclictest2035-21kworker/0:019:19:120
179199190,15cyclictest31004-21/usr/sbin/munin20:10:010
1802991818,0cyclictest0-21swapper/320:46:223
1802991818,0cyclictest0-21swapper/319:13:373
1796991818,0cyclictest0-21swapper/220:03:382
1796991818,0cyclictest0-21swapper/219:28:382
1796991818,0cyclictest0-21swapper/219:18:382
1793991818,0cyclictest0-21swapper/120:35:001
1793991818,0cyclictest0-21swapper/119:18:381
1791991818,0cyclictest0-21swapper/019:43:400
1791991816,1cyclictest13039-21tr20:40:010
1802991717,0cyclictest0-21swapper/319:48:393
1796991717,0cyclictest0-21swapper/220:58:392
1796991715,1cyclictest1499-21apt-get20:18:392
1796991615,0cyclictest0-21swapper/220:23:362
1793991616,0cyclictest0-21swapper/120:53:421
1793991616,0cyclictest0-21swapper/120:43:401
179399160,5cyclictest0-21swapper/119:58:391
1791991616,0cyclictest0-21swapper/020:03:370
1791991616,0cyclictest0-21swapper/019:23:370
1802991515,0cyclictest0-21swapper/319:38:393
1796991515,0cyclictest0-21swapper/219:53:392
1796991513,1cyclictest0-21swapper/220:38:452
179699150,10cyclictest313-21systemd-journal20:25:522
1793991515,0cyclictest4195-21munin-run19:15:011
179399150,1cyclictest0-21swapper/119:50:011
180299140,14cyclictest0-21swapper/320:52:233
180299140,14cyclictest0-21swapper/320:25:083
180299140,0cyclictest0-21swapper/320:21:073
180299140,0cyclictest0-21swapper/320:05:153
179699140,14cyclictest0-21swapper/220:45:272
179699140,14cyclictest0-21swapper/220:32:292
179699140,14cyclictest0-21swapper/219:45:572
179399140,14cyclictest0-21swapper/120:12:071
179399140,14cyclictest0-21swapper/119:30:021
179199140,14cyclictest0-21swapper/020:30:520
179199140,14cyclictest0-21swapper/020:17:180
179199140,0cyclictest0-21swapper/020:12:430
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional