You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 14:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Apr 16, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
178362235160,25sleep00-21swapper/007:07:460
1714922240,6sleep2311ktimersoftd/207:05:212
177192202166,24sleep10-21swapper/107:06:141
177372200164,24sleep30-21swapper/307:06:283
1207721250,1sleep00-21swapper/008:05:250
18103998784,2cyclictest0-21swapper/207:27:242
18103996663,2cyclictest0-21swapper/208:27:452
320812650,3sleep20-21swapper/208:50:262
13522650,2sleep30-21swapper/310:10:123
29332570,4sleep339-21rcuc/311:25:263
18103995047,2cyclictest0-21swapper/211:34:582
249002470,3sleep01809199cyclictest08:35:180
1809899324,17cyclictest23997-21sshd08:35:051
1809899310,23cyclictest0-21swapper/110:44:501
1810999301,10cyclictest0-21swapper/311:14:453
18098993011,10cyclictest0-21swapper/109:45:011
66812290,1sleep10-21swapper/110:20:231
1809899291,8cyclictest25640-21kworker/u8:210:49:261
1809899291,17cyclictest0-21swapper/112:10:091
316152280,3sleep011-21rcuc/011:20:000
1809899281,18cyclictest0-21swapper/108:30:201
1809899281,17cyclictest0-21swapper/110:56:561
1809899281,16cyclictest0-21swapper/111:29:541
1809899281,11cyclictest28551-21cpuspeed_turbos08:45:121
1809899281,11cyclictest14011-21perf10:40:021
1809899281,10cyclictest0-21swapper/112:15:221
267712270,3sleep30-21swapper/312:20:193
1810999279,1cyclictest0-21swapper/310:05:163
1810999271,23cyclictest11442-21grep11:45:213
1810999271,22cyclictest5978-21head09:05:223
1810999270,24cyclictest10220-21needreboot09:15:193
1809899274,11cyclictest13617-21sshd09:25:091
1809899272,3cyclictest25640-21kworker/u8:210:00:361
1809899272,13cyclictest753-21awk10:10:001
1809899271,17cyclictest13013-21users09:20:261
1809899271,17cyclictest13013-21users09:20:261
1809899271,17cyclictest0-21swapper/112:32:001
1809899271,16cyclictest0-21swapper/107:50:211
1809899271,15cyclictest0-21swapper/111:03:141
1809899270,7cyclictest16167-21kworker/u8:011:46:161
1809199271,6cyclictest0-21swapper/010:48:090
1810999261,23cyclictest19084-21sshd09:35:223
1809899269,9cyclictest0-21swapper/109:00:131
1809899267,11cyclictest0-21swapper/109:30:181
1809899267,10cyclictest0-21swapper/109:58:141
1809899264,9cyclictest0-21swapper/111:57:571
1809899264,16cyclictest25640-21kworker/u8:210:55:021
1809899261,16cyclictest0-21swapper/111:39:581
1809899261,16cyclictest0-21swapper/111:39:581
1809899261,15cyclictest32173-21diskstats11:20:151
1809899261,15cyclictest0-21swapper/112:25:491
1809899261,15cyclictest0-21swapper/110:33:181
1809899261,15cyclictest0-21swapper/110:33:181
1809199261,6cyclictest0-21swapper/011:52:130
1809199260,13cyclictest2058-21munin-run12:40:010
1810999258,4cyclictest25238-21/usr/sbin/munin11:05:213
1810999251,22cyclictest0-21swapper/311:56:333
1810999251,22cyclictest0-21swapper/311:30:213
1810999251,11cyclictest319-21/usr/sbin/munin08:55:143
1809899259,9cyclictest0-21swapper/107:10:291
1809899257,10cyclictest0-21swapper/111:53:181
1809899252,20cyclictest1145-21tr10:10:131
1809899251,5cyclictest0-21swapper/110:28:231
18098992513,3cyclictest0-21swapper/108:15:131
18098992512,3cyclictest1342-21switchtime12:35:261
1809899251,16cyclictest0-21swapper/109:53:041
1809899251,16cyclictest0-21swapper/109:15:241
1809899251,16cyclictest0-21swapper/108:58:391
1809899251,15cyclictest0-21swapper/109:35:351
1809899251,15cyclictest0-21swapper/109:13:571
1809899251,15cyclictest0-21swapper/108:04:261
1809899250,16cyclictest0-21swapper/112:00:221
1809199251,21cyclictest30377-21idleruntime-cro08:50:010
217202241,3sleep321869-21hddtemp_smartct07:15:163
18109992413,7cyclictest0-21swapper/312:28:103
1810999241,21cyclictest0-21swapper/310:59:483
1810999241,21cyclictest0-21swapper/310:47:163
1810999241,21cyclictest0-21swapper/310:41:263
1810999241,20cyclictest0-21swapper/310:32:513
1810999241,20cyclictest0-21swapper/310:32:513
1810999241,20cyclictest0-21swapper/310:16:223
1810999241,18cyclictest0-21swapper/309:11:463
1810999241,12cyclictest6798-21diskmemload10:36:253
1810999241,11cyclictest3816-21sshd07:48:023
1810999241,11cyclictest31297-21ntp_offset12:30:203
1809899249,9cyclictest0-21swapper/111:32:311
1809899249,9cyclictest0-21swapper/107:40:151
1809899248,9cyclictest0-21swapper/112:06:591
1809899248,8cyclictest0-21swapper/108:11:481
1809899248,8cyclictest0-21swapper/108:11:481
1809899243,9cyclictest28500-21sshd11:10:221
1809899242,12cyclictest29755-21munin-run07:35:011
18098992415,3cyclictest0-21swapper/107:28:461
18098992414,2cyclictest0-21swapper/108:20:141
1809899241,17cyclictest0-21swapper/111:40:041
1809899241,16cyclictest0-21swapper/112:20:171
1809899241,16cyclictest0-21swapper/108:54:081
1809899241,15cyclictest0-21swapper/108:06:121
1809899241,14cyclictest0-21swapper/110:19:121
1809899241,10cyclictest26342-21cpuspeed_turbos08:40:131
1809899240,16cyclictest0-21swapper/111:05:081
1810999232,3cyclictest6760-21diskstats07:55:143
1810999232,10cyclictest8608-21/usr/sbin/munin08:00:233
1810999231,3cyclictest0-21swapper/310:51:073
1810999231,20cyclictest0-21swapper/312:01:453
1810999231,20cyclictest0-21swapper/311:40:403
1810999231,20cyclictest0-21swapper/311:17:373
1810999231,20cyclictest0-21swapper/309:58:283
1810999231,20cyclictest0-21swapper/308:15:213
1810399236,3cyclictest10830-21cut10:30:202
1810399236,3cyclictest10830-21cut10:30:202
1810399231,9cyclictest3012-21latency_hist10:15:012
1809899233,14cyclictest23127-21sshd07:19:461
1809899231,3cyclictest25365-21perf07:25:011
1809899231,14cyclictest0-21swapper/109:45:581
1809899230,16cyclictest0-21swapper/107:56:491
1809899230,15cyclictest0-21swapper/111:16:071
1809199237,4cyclictest0-21swapper/010:40:010
1809199234,3cyclictest27032-21kworker/u8:211:11:340
1809199232,5cyclictest0-21swapper/009:25:210
1809199231,5cyclictest16167-21kworker/u8:011:58:540
1809199231,5cyclictest0-21swapper/011:41:590
1809199231,5cyclictest0-21swapper/009:41:540
1810999222,6cyclictest13719-21ntp_states11:50:213
1810999222,17cyclictest30065-21ntp_states10:00:193
1810999221,4cyclictest4817-21grep07:50:153
1810999221,20cyclictest0-21swapper/309:45:343
1810999221,19cyclictest0-21swapper/312:05:223
1810999221,10cyclictest8694-21ntpq10:25:203
1810999221,10cyclictest0-21swapper/310:22:083
1810999220,20cyclictest0-21swapper/309:30:153
1810999220,20cyclictest0-21swapper/307:44:313
1810399221,7cyclictest17792-21perf09:35:012
1810399221,12cyclictest0-21swapper/209:13:042
1810399220,20cyclictest0-21swapper/209:35:232
1809199222,8cyclictest29677-21kworker/0:112:26:310
1809199221,9cyclictest0-21swapper/011:06:360
1809199221,5cyclictest18941-21kworker/u8:312:31:210
1810999215,3cyclictest0-21swapper/312:36:373
1810999214,4cyclictest21807-21users09:40:263
1810999212,3cyclictest31827-21/usr/sbin/munin11:20:143
1810999211,8cyclictest0-21swapper/308:14:293
1810999211,8cyclictest0-21swapper/308:14:293
1810999211,7cyclictest25883-21sshd08:39:003
18109992116,3cyclictest31511-21tail07:35:273
1810999211,10cyclictest0-21swapper/312:10:393
1810999211,10cyclictest0-21swapper/308:50:123
1810999210,2cyclictest0-21swapper/307:25:203
1810999210,19cyclictest0-21swapper/309:50:253
1810999210,10cyclictest0-21swapper/311:36:593
1810999210,10cyclictest0-21swapper/311:36:593
1810399216,8cyclictest87650irq/129-enp1s0-07:12:272
1810399211,9cyclictest30979-21latency_hist10:05:012
18103992115,3cyclictest30045-21grep11:15:162
18103992114,4cyclictest17168-21tr12:00:122
1810399211,10cyclictest0-21swapper/211:20:082
1809899211,15cyclictest0-21swapper/109:07:321
1809899211,15cyclictest0-21swapper/107:35:421
1809899211,10cyclictest21108-21sed08:25:261
1809199216,6cyclictest6798-21diskmemload11:04:200
1809199216,4cyclictest23006-21head09:45:160
1809199216,4cyclictest17175-21grep08:20:020
1809199216,11cyclictest87550irq/128-enp1s0-10:23:150
1809199215,4cyclictest25363-21awk07:25:010
1809199213,13cyclictest87550irq/128-enp1s0-10:56:530
1809199213,12cyclictest87550irq/128-enp1s0-10:15:170
1809199211,4cyclictest0-21swapper/009:51:040
1809199211,3cyclictest0-21swapper/009:22:450
1809199211,3cyclictest0-21swapper/009:22:450
1809199211,2cyclictest0-21swapper/010:11:430
18091992111,3cyclictest87550irq/128-enp1s0-12:12:500
1809199211,12cyclictest18634-21/usr/sbin/munin10:50:140
1809199210,5cyclictest18941-21kworker/u8:312:23:110
1810999202,15cyclictest23389-21/usr/sbin/munin07:20:133
1810999201,9cyclictest0-21swapper/309:27:513
1810999201,8cyclictest0-21swapper/312:16:213
1810999201,4cyclictest29746-21latency_hist07:35:023
1810999201,3cyclictest23540-21fschecks_time11:00:163
1810999201,16cyclictest12992-21awk09:20:263
1810999201,16cyclictest12992-21awk09:20:263
1810399205,3cyclictest31234-21needreboot12:30:202
1810399205,3cyclictest18404-21strings08:20:192
1810399204,4cyclictest14995-21if_enp2s010:40:182
1810399202,6cyclictest23032-21/usr/sbin/munin11:00:222
1810399201,9cyclictest9247-21grep11:40:202
1810399201,7cyclictest18403-21latency_hist10:50:012
1810399201,5cyclictest909-21sssd_nss09:20:232
1810399201,5cyclictest909-21sssd_nss09:20:232
1810399201,5cyclictest0-21swapper/210:53:112
18103992015,3cyclictest27924-21ls12:25:012
18103992013,4cyclictest29907-21grep08:45:262
1810399201,17cyclictest0-21swapper/211:05:232
1809899201,10cyclictest0-21swapper/107:49:161
1809199205,4cyclictest0-21swapper/010:27:210
1809199203,13cyclictest87550irq/128-enp1s0-12:16:200
1809199202,5cyclictest0-21swapper/011:22:440
1809199202,3cyclictest11441-21strings11:45:200
1809199201,9cyclictest0-21swapper/010:30:180
1809199201,9cyclictest0-21swapper/010:30:180
1809199201,6cyclictest29814-21irqstats10:00:170
18091992012,2cyclictest87550irq/128-enp1s0-09:13:590
1809199201,11cyclictest32176-21/usr/sbin/munin07:40:180
1809199201,11cyclictest15818-21/usr/sbin/munin09:30:150
1809199201,10cyclictest0-21swapper/009:15:330
1810999191,5cyclictest28736-21df_inode08:45:133
1810999191,4cyclictest18299-21/usr/sbin/munin07:10:243
1810399195,4cyclictest6195-21if_enp2s010:20:152
1810399195,3cyclictest32397-21cat07:40:112
1810399193,13cyclictest16921-21cat12:00:012
1810399192,14cyclictest3266-21gltestperf09:00:142
1810399191,8cyclictest0-21swapper/210:19:502
1810399191,8cyclictest0-21swapper/210:08:052
1810399191,7cyclictest0-21swapper/209:55:132
1810399191,6cyclictest9420-21irqstats08:00:162
1810399191,6cyclictest6798-21diskmemload09:44:282
1810399191,6cyclictest29746-21latency_hist07:35:012
1810399191,6cyclictest23389-21/usr/sbin/munin07:20:192
1810399191,6cyclictest0-21swapper/210:58:492
1810399191,5cyclictest0-21swapper/212:37:132
1810399191,4cyclictest87650irq/129-enp1s0-12:15:112
18103991913,3cyclictest5465-21sed09:05:172
1810399191,15cyclictest7582-21ps07:55:232
1809199199,3cyclictest87550irq/128-enp1s0-12:08:440
1809199192,6cyclictest1985-21grep08:55:250
1809199191,15cyclictest27155-21diskstats09:55:140
1809199191,11cyclictest18299-21/usr/sbin/munin07:10:180
1809199191,10cyclictest5860-21sshd07:51:010
1809199191,10cyclictest3495-21sshd11:28:220
1809199191,10cyclictest14233-21/usr/sbin/munin10:40:230
236972181,3sleep20-21swapper/208:34:092
1810999182,3cyclictest0-21swapper/308:07:523
1810999182,15cyclictest0-21swapper/308:40:333
18109991813,3cyclictest18477-21ntp_states08:20:213
1810399184,3cyclictest12050irq/123-ahci[0008:40:012
1810399182,3cyclictest87350irq/126-enp1s0-11:10:262
1810399181,9cyclictest0-21swapper/209:16:482
1810399181,8cyclictest0-21swapper/212:28:222
1810399181,7cyclictest0-21swapper/212:13:272
1810399181,5cyclictest14619-21latency_hist11:55:012
1810399181,5cyclictest0-21swapper/212:09:452
1810399181,5cyclictest0-21swapper/211:39:372
1810399181,5cyclictest0-21swapper/211:39:372
18103991814,2cyclictest25276-21gltestperf09:50:132
1810399181,3cyclictest87350irq/126-enp1s0-10:26:042
18103991813,3cyclictest13777-21tr09:25:112
1810399181,16cyclictest0-21swapper/208:17:072
1809199187,3cyclictest808-21snmpd08:32:480
1809199185,9cyclictest87550irq/128-enp1s0-12:01:460
1809199185,6cyclictest87550irq/128-enp1s0-07:55:200
1809199185,4cyclictest3561-21kworker/u8:211:32:130
1809199184,5cyclictest579-21systemd-journal08:45:010
1809199184,2cyclictest12050irq/123-ahci[0007:40:010
1809199182,5cyclictest21581-21munin-run08:30:010
1809199182,4cyclictest32202-21strings10:05:210
1809199182,4cyclictest0-21swapper/009:35:140
1809199181,5cyclictest29766-21awk07:35:010
1809199181,10cyclictest17407-21/usr/sbin/munin08:20:180
1810999173,3cyclictest87650irq/129-enp1s0-09:00:233
1810999171,4cyclictest21413-21sshd08:28:013
18109991712,3cyclictest22418-21grep08:30:163
1810399174,3cyclictest32-21ksoftirqd/211:45:082
1810399172,7cyclictest0-21swapper/211:27:402
1810399171,8cyclictest0-21swapper/210:35:502
1810399171,5cyclictest319-21/usr/sbin/munin08:55:232
1810399171,4cyclictest23328-21memory09:45:182
1810399171,3cyclictest11333-21grep08:05:162
18103991710,4cyclictest26849-21tail08:40:162
1809199174,4cyclictest2813-21cat09:00:100
1809199174,3cyclictest6179-21tr11:35:130
1809199174,3cyclictest6179-21tr11:35:130
1809199172,3cyclictest29517-21kworker/0:307:45:110
1809199172,3cyclictest27560-21munin-run07:30:010
1809199171,5cyclictest31732-21pmu-power08:50:220
1809199171,3cyclictest0-21swapper/008:12:010
1809199171,3cyclictest0-21swapper/008:12:010
18091991711,3cyclictest4834-21/usr/sbin/munin09:05:210
18091991710,4cyclictest22296-21/usr/sbin/munin07:15:220
1810399169,4cyclictest0-21swapper/207:50:142
1810399165,2cyclictest87550irq/128-enp1s0-08:10:112
1810399165,2cyclictest87550irq/128-enp1s0-08:10:112
1810399161,14cyclictest0-21swapper/207:37:562
18103991610,3cyclictest2548-21fschecks_time07:45:142
18091991611,3cyclictest8608-21/usr/sbin/munin08:00:130
18103991510,3cyclictest20966-21cut07:15:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional