You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 20:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Tue May 12, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
192372237162,25sleep00-21swapper/007:09:360
190922236163,24sleep20-21swapper/207:07:412
189622202166,24sleep30-21swapper/307:06:013
189552200163,24sleep10-21swapper/107:05:551
1199721460,4sleep11936499cyclictest10:30:181
19370991100,2cyclictest0-21swapper/209:14:312
19370997774,2cyclictest0-21swapper/208:23:542
19370997571,3cyclictest0-21swapper/211:25:272
1937799391,28cyclictest0-21swapper/308:01:143
1937099321,14cyclictest0-21swapper/207:39:522
19370993110,11cyclictest0-21swapper/210:05:182
19360993013,15cyclictest3062-21basename07:45:010
1937799291,21cyclictest0-21swapper/311:56:103
1937799291,11cyclictest0-21swapper/310:45:583
1937799291,10cyclictest0-21swapper/308:07:163
19370992915,8cyclictest0-21swapper/207:10:482
1937099291,15cyclictest4189-21memory07:45:202
1936099294,11cyclictest16867-21awk09:30:010
19370992816,3cyclictest0-21swapper/208:47:402
1937099280,12cyclictest0-21swapper/211:17:162
1936499280,20cyclictest0-21swapper/111:51:381
1937799271,9cyclictest0-21swapper/307:29:183
1937799271,13cyclictest0-21swapper/311:39:283
1937799271,11cyclictest0-21swapper/312:21:063
1937799271,11cyclictest0-21swapper/311:02:403
1937799271,11cyclictest0-21swapper/311:02:393
1937799271,10cyclictest0-21swapper/310:36:023
1937799270,13cyclictest0-21swapper/308:50:183
1937099279,3cyclictest17429-21diskstats09:30:142
1937099279,2cyclictest0-21swapper/207:30:242
1937099279,2cyclictest0-21swapper/207:30:242
1937099271,3cyclictest27852-21if_enp1s012:20:162
19370992712,3cyclictest15248-21switchtime11:50:242
1937799261,17cyclictest0-21swapper/310:40:303
1937799261,17cyclictest0-21swapper/309:30:563
1937799261,13cyclictest0-21swapper/309:20:093
1937799261,13cyclictest0-21swapper/307:50:583
1937799261,12cyclictest827-21runrttasks10:24:383
1937799261,12cyclictest0-21swapper/309:35:323
1937799261,11cyclictest0-21swapper/309:46:223
1937799261,11cyclictest0-21swapper/307:10:303
1937799261,10cyclictest0-21swapper/311:48:263
1937799260,11cyclictest0-21swapper/311:30:183
1937799260,11cyclictest0-21swapper/309:00:493
1937799260,10cyclictest0-21swapper/311:09:083
1937799260,10cyclictest0-21swapper/309:58:503
1937099266,2cyclictest0-21swapper/211:20:142
1937099261,15cyclictest0-21swapper/209:35:162
19370992610,3cyclictest27801-21diskstats08:40:142
19370992610,3cyclictest25712-21ntp_states07:20:222
1936099263,11cyclictest22848-21perf08:30:010
1937799251,9cyclictest0-21swapper/309:09:383
1937799251,7cyclictest0-21swapper/309:16:223
1937799251,11cyclictest0-21swapper/308:57:183
1937799251,10cyclictest0-21swapper/307:30:523
1937799251,10cyclictest0-21swapper/307:30:523
1937799250,11cyclictest0-21swapper/309:10:583
1937799250,10cyclictest0-21swapper/312:17:143
1937799250,10cyclictest0-21swapper/312:17:143
1937099259,3cyclictest0-21swapper/207:40:022
1937099251,11cyclictest0-21swapper/211:45:042
19370992510,3cyclictest0-21swapper/209:05:382
19370992510,3cyclictest0-21swapper/209:01:542
19370992510,3cyclictest0-21swapper/207:54:062
1936499251,5cyclictest21166-21gltestperf12:05:161
1936499250,8cyclictest1999-21kworker/u8:110:17:171
1936099251,12cyclictest8685-21munin-run10:25:010
1936099251,12cyclictest11-21rcuc/007:40:000
1937799241,9cyclictest0-21swapper/307:42:123
1937799241,10cyclictest0-21swapper/311:19:583
1937799241,10cyclictest0-21swapper/310:10:343
1937799241,10cyclictest0-21swapper/310:09:163
1937799240,9cyclictest0-21swapper/312:08:543
1937799240,11cyclictest0-21swapper/311:20:023
1937799240,11cyclictest0-21swapper/310:26:503
1937099249,3cyclictest0-21swapper/208:26:262
1937099249,3cyclictest0-21swapper/208:00:302
1937099248,3cyclictest0-21swapper/209:56:422
1937099247,3cyclictest0-21swapper/211:41:182
1937099246,3cyclictest0-21swapper/209:41:362
19370992411,1cyclictest0-21swapper/211:00:582
19370992411,1cyclictest0-21swapper/211:00:582
1937099241,10cyclictest0-21swapper/210:41:042
19370992410,3cyclictest0-21swapper/212:28:462
19370992410,3cyclictest0-21swapper/210:26:452
19370992410,3cyclictest0-21swapper/209:45:182
1936499241,11cyclictest0-21swapper/111:55:341
1936099248,13cyclictest2710-21munin-run11:25:010
1936099241,11cyclictest26629-21awk07:25:010
252132230,2sleep20-21swapper/208:35:012
221722230,14sleep30-21swapper/310:55:123
1937799231,9cyclictest0-21swapper/312:31:163
1937799231,5cyclictest0-21swapper/311:27:003
1937799231,10cyclictest0-21swapper/312:00:383
1937799230,9cyclictest0-21swapper/309:51:543
1937799230,8cyclictest0-21swapper/311:41:023
1937799230,11cyclictest0-21swapper/310:54:123
1937099239,3cyclictest0-21swapper/212:00:222
1937099239,3cyclictest0-21swapper/211:38:302
1937099239,3cyclictest0-21swapper/208:10:022
1937099238,3cyclictest0-21swapper/211:56:302
1937099238,3cyclictest0-21swapper/210:22:532
1937099238,3cyclictest0-21swapper/210:02:142
1937099238,1cyclictest0-21swapper/207:25:182
1937099236,5cyclictest0-21swapper/212:30:402
1937099235,3cyclictest0-21swapper/212:19:022
1937099235,3cyclictest0-21swapper/212:19:012
1936499238,3cyclictest827-21runrttasks08:40:411
19364992316,4cyclictest20890-21strings10:50:201
1937799228,3cyclictest0-21swapper/309:29:323
1937799221,9cyclictest0-21swapper/308:39:163
1937799221,9cyclictest0-21swapper/307:46:103
1937799220,9cyclictest0-21swapper/312:10:523
1937799220,10cyclictest0-21swapper/312:39:183
1937799220,10cyclictest0-21swapper/310:30:063
1937799220,10cyclictest0-21swapper/308:45:223
1937099229,3cyclictest0-21swapper/211:34:142
1937099228,3cyclictest0-21swapper/210:59:182
1937099228,3cyclictest0-21swapper/210:46:302
1937099228,3cyclictest0-21swapper/210:11:382
1937099228,3cyclictest0-21swapper/208:55:022
1937099227,3cyclictest0-21swapper/207:17:442
1937099226,3cyclictest0-21swapper/212:09:222
1937099220,9cyclictest0-21swapper/210:35:182
1936499227,7cyclictest0-21swapper/110:27:321
1936499224,2cyclictest32689-21kworker/u8:011:22:451
1936499223,10cyclictest23949-21sshd10:58:381
1936499222,14cyclictest87550irq/128-enp1s0-12:37:241
1936499221,3cyclictest5211-21/usr/sbin/munin11:30:121
1936499221,11cyclictest0-21swapper/111:29:451
1936499221,11cyclictest0-21swapper/109:28:011
1936499221,11cyclictest0-21swapper/108:39:541
1936499220,6cyclictest0-21swapper/110:43:241
1936099221,7cyclictest0-21swapper/011:42:400
19360992216,4cyclictest7-21ksoftirqd/007:30:170
19360992216,4cyclictest7-21ksoftirqd/007:30:170
1936099221,5cyclictest0-21swapper/009:37:350
1936099221,14cyclictest0-21swapper/010:44:260
1937799217,3cyclictest0-21swapper/310:18:403
1937799211,9cyclictest0-21swapper/308:31:043
1937799211,11cyclictest0-21swapper/310:01:363
1937799210,8cyclictest0-21swapper/309:42:063
1937099218,3cyclictest0-21swapper/210:32:462
1937099217,3cyclictest0-21swapper/212:14:462
1937099216,3cyclictest0-21swapper/209:15:282
1937099211,11cyclictest0-21swapper/211:10:042
1936499219,2cyclictest15239-21needreboot08:10:191
1936499211,9cyclictest0-21swapper/107:25:171
1936499211,6cyclictest0-21swapper/110:13:331
1936499211,12cyclictest0-21swapper/112:03:331
1936499211,11cyclictest25266-21diskstats12:15:141
1936499211,11cyclictest25266-21diskstats12:15:141
1936499211,11cyclictest25032-21grep07:20:161
1936499211,11cyclictest0-21swapper/112:14:561
1936499211,10cyclictest0-21swapper/111:01:181
1936499211,10cyclictest0-21swapper/111:01:181
1936499210,5cyclictest17248-21kworker/u8:110:46:031
19360992115,3cyclictest7-21ksoftirqd/009:25:010
1936099211,12cyclictest5125-21/usr/sbin/munin11:30:110
1937799207,3cyclictest29869-21perl12:25:143
1937799206,2cyclictest0-21swapper/308:22:163
1937799201,11cyclictest27452-21/usr/sbin/munin08:40:143
1937799200,11cyclictest0-21swapper/311:53:243
1937799200,11cyclictest0-21swapper/311:13:003
1937099207,3cyclictest0-21swapper/208:35:142
1937099206,3cyclictest0-21swapper/211:05:562
1937099201,4cyclictest0-21swapper/210:17:342
1936499204,9cyclictest0-21swapper/111:36:131
1936499201,7cyclictest0-21swapper/109:51:421
19364992016,3cyclictest0-21swapper/108:55:131
1936499201,5cyclictest0-21swapper/109:42:221
1936499201,3cyclictest24735-21ntpq09:45:211
1936499201,3cyclictest13950-21swap09:20:241
1936499201,2cyclictest29134-21ntpq09:55:201
1936499201,2cyclictest21409-21sshd07:13:381
1936499201,18cyclictest0-21swapper/108:27:581
1936499201,17cyclictest0-21swapper/112:30:111
1936499201,17cyclictest0-21swapper/109:35:491
1936499201,17cyclictest0-21swapper/107:38:431
1936499201,11cyclictest32041-21ls08:50:131
1936499201,11cyclictest30526-21latency08:45:181
1936499201,11cyclictest0-21swapper/110:21:381
1936499201,10cyclictest27923-21irqstats12:20:171
1936499200,18cyclictest0-21swapper/107:15:021
1936099209,3cyclictest10490-21/usr/sbin/munin09:15:170
1936099203,5cyclictest27567-21fschecks_count12:20:140
19360992016,2cyclictest11-21rcuc/010:30:020
1936099201,5cyclictest0-21swapper/011:07:150
1936099201,11cyclictest7669-21/usr/sbin/munin07:55:220
1936099201,11cyclictest17081-21/usr/sbin/munin09:30:190
48862190,15sleep00-21swapper/011:30:020
1937799191,10cyclictest24647-21/usr/sbin/munin07:20:163
1937799191,10cyclictest18078-21sshd08:15:383
1937099194,3cyclictest0-21swapper/207:55:142
1937099191,13cyclictest0-21swapper/209:27:202
1937099191,11cyclictest0-21swapper/210:51:442
1936499198,3cyclictest18175-21open_inodes09:30:221
1936499191,8cyclictest0-21swapper/111:42:031
19364991916,2cyclictest0-21swapper/107:56:161
1936499191,16cyclictest0-21swapper/110:35:041
1936499191,16cyclictest0-21swapper/107:31:481
1936499191,16cyclictest0-21swapper/107:31:481
1936499191,10cyclictest28773-21cpuspeed_turbos11:10:121
1936499191,10cyclictest12980-21load08:05:181
1936499191,10cyclictest0-21swapper/111:17:481
1936499191,10cyclictest0-21swapper/109:16:511
1936499190,6cyclictest0-21swapper/109:10:591
1936499190,15cyclictest87550irq/128-enp1s0-12:26:051
1936099198,3cyclictest24297-21/usr/sbin/munin11:00:230
1936099198,3cyclictest24297-21/usr/sbin/munin11:00:230
1936099198,3cyclictest16460-21/usr/sbin/munin08:15:160
1936099191,6cyclictest0-21swapper/010:01:180
1936099191,5cyclictest0-21swapper/012:38:490
1936099191,5cyclictest0-21swapper/010:45:210
19360991912,4cyclictest6756-21tail09:05:150
1936099191,10cyclictest8938-21sshd11:35:330
277782181,2sleep20-21swapper/209:54:202
1937099181,14cyclictest0-21swapper/209:23:022
1936499188,3cyclictest4348-21ntpq07:45:231
1936499181,8cyclictest0-21swapper/110:05:481
19364991815,2cyclictest0-21swapper/109:08:051
19364991815,2cyclictest0-21swapper/108:18:511
1936499181,16cyclictest0-21swapper/111:46:361
1936499181,16cyclictest0-21swapper/108:20:031
1936499181,10cyclictest3886-21/usr/sbin/munin09:00:131
1936499181,10cyclictest10588-21if_enp1s008:00:181
1936499181,10cyclictest0-21swapper/110:00:071
1936499181,10cyclictest0-21swapper/108:30:161
1936499181,10cyclictest0-21swapper/107:50:141
1936499180,17cyclictest0-21swapper/111:05:251
1936099187,3cyclictest1690-21/usr/sbin/munin08:55:250
1936099184,3cyclictest17638-1kworker/u9:009:11:040
1936099182,4cyclictest26892-21munin-run12:20:010
1936099182,4cyclictest26892-21munin-run12:20:010
1936099181,7cyclictest0-21swapper/011:46:150
1936099181,5cyclictest30639-21runrttasks11:14:480
1936099181,4cyclictest17638-1kworker/u9:010:30:030
19360991813,3cyclictest1086-21needreboot10:05:210
1936099181,10cyclictest13304-21/usr/sbin/munin10:35:210
1936099181,10cyclictest0-21swapper/008:45:130
311222170,2sleep00-21swapper/012:28:050
201102171,2sleep00-21swapper/012:02:450
1937799171,9cyclictest0-21swapper/308:10:163
1937799171,14cyclictest0-21swapper/307:17:373
19377991711,3cyclictest21522-21sed08:25:163
19370991714,2cyclictest18441-21awk08:20:002
1937099170,11cyclictest0-21swapper/212:38:522
1936499171,2cyclictest0-21swapper/107:42:051
1936099179,2cyclictest0-21swapper/007:10:270
1936099172,9cyclictest0-21swapper/010:58:500
1936099172,7cyclictest0-21swapper/012:05:230
1936099171,9cyclictest0-21swapper/010:14:350
1936099171,8cyclictest0-21swapper/008:53:380
1936099171,8cyclictest0-21swapper/007:46:380
1936099171,7cyclictest27452-21/usr/sbin/munin08:40:240
1936099171,7cyclictest23681-21/usr/sbin/munin09:45:160
1936099171,6cyclictest27371-21unixbench_multi09:50:260
1936099171,6cyclictest0-21swapper/011:57:390
1936099171,6cyclictest0-21swapper/011:50:200
1936099171,6cyclictest0-21swapper/010:50:460
1936099171,4cyclictest18654-21/usr/sbin/munin08:20:220
1936099171,4cyclictest0-21swapper/007:25:160
1936099171,3cyclictest28074-21/usr/sbin/munin09:55:180
1936099171,3cyclictest0-21swapper/012:32:460
19360991713,3cyclictest0-21swapper/008:35:060
1936099171,14cyclictest0-21swapper/009:01:530
19360991711,3cyclictest9967-21grep08:00:140
19360991711,3cyclictest22869-21fschecks_count07:15:150
1936099171,11cyclictest0-21swapper/012:10:500
19377991611,3cyclictest7669-21/usr/sbin/munin07:55:183
19377991611,3cyclictest32638-21sed07:35:263
19370991612,3cyclictest0-21swapper/208:06:552
1936099165,4cyclictest11-21rcuc/008:32:560
1936099161,6cyclictest0-21swapper/010:19:580
19360991612,3cyclictest7-21ksoftirqd/008:10:110
19360991611,3cyclictest7322-21sshd07:53:380
19360991611,3cyclictest30896-21/usr/sbin/munin11:15:190
19360991611,3cyclictest21480-21/usr/sbin/munin09:40:170
19370991510,3cyclictest32215-21forks08:50:152
1936099150,14cyclictest0-21swapper/008:06:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional