You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 10:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu May 07, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
236942234163,57sleep10-21swapper/119:08:431
236432232163,56sleep00-21swapper/019:08:020
237772202166,24sleep20-21swapper/219:09:492
235542202166,24sleep30-21swapper/319:06:533
23889997167,3cyclictest0-21swapper/220:35:222
2389499443,5cyclictest9373-21df22:15:123
2389499361,6cyclictest808-21snmpd21:39:463
2389499352,5cyclictest0-21swapper/320:50:013
2389499342,5cyclictest961-21munin-run19:30:023
2389499331,19cyclictest0-21swapper/319:12:353
2389499331,18cyclictest0-21swapper/319:30:483
2387799337,12cyclictest29260-21perf00:15:010
2389499328,7cyclictest87350irq/126-enp1s0-23:45:263
2389499322,10cyclictest12589-21diskmemload23:42:373
2389499322,10cyclictest12589-21diskmemload23:42:373
2389499320,14cyclictest0-21swapper/319:44:243
2388999321,13cyclictest0-21swapper/200:00:022
2389499311,13cyclictest0-21swapper/319:15:073
2388999311,19cyclictest0-21swapper/219:49:192
2388999311,14cyclictest0-21swapper/222:30:132
2388999311,14cyclictest0-21swapper/222:30:132
2388499319,6cyclictest28799-40dnf20:30:241
2389499301,14cyclictest0-21swapper/320:52:233
2389499301,14cyclictest0-21swapper/320:40:553
2389499301,13cyclictest0-21swapper/323:52:403
2389499301,11cyclictest6953-21ntpq00:35:193
2389499300,17cyclictest0-21swapper/322:59:273
2389499300,13cyclictest0-21swapper/322:34:153
2389499300,13cyclictest0-21swapper/322:34:153
2389499300,12cyclictest0-21swapper/320:29:073
2388999301,14cyclictest0-21swapper/220:00:232
2387799307,11cyclictest28957-21perf19:20:010
228612300,4sleep22388999cyclictest22:45:142
2389499299,3cyclictest0-21swapper/321:27:043
2389499298,3cyclictest0-21swapper/319:24:443
23894992912,3cyclictest27946-21hddtemp_smartct00:10:163
2389499291,14cyclictest17887-21hddtemp_smartct21:20:163
2389499291,14cyclictest17887-21hddtemp_smartct21:20:153
23894992911,3cyclictest10339-21diskstats19:50:123
2389499291,13cyclictest0-21swapper/322:51:193
2389499291,13cyclictest0-21swapper/322:26:113
2389499291,13cyclictest0-21swapper/321:45:363
2389499291,12cyclictest0-21swapper/321:44:523
2389499291,12cyclictest0-21swapper/319:58:203
2389499290,19cyclictest0-21swapper/320:10:473
2389499290,15cyclictest0-21swapper/300:15:163
2389499290,13cyclictest0-21swapper/300:04:563
2387799294,12cyclictest11011-21munin-run22:20:010
2389499281,14cyclictest0-21swapper/323:37:553
2389499281,13cyclictest0-21swapper/323:33:073
2389499280,13cyclictest0-21swapper/321:55:283
2389499280,12cyclictest0-21swapper/320:59:193
2388999281,9cyclictest24178-21/usr/sbin/munin21:35:162
2388999281,16cyclictest0-21swapper/223:36:322
2388999281,12cyclictest0-21swapper/220:59:262
2388999281,12cyclictest0-21swapper/200:38:172
2388999281,11cyclictest0-21swapper/220:53:082
2388999280,13cyclictest0-21swapper/200:20:172
2387799283,12cyclictest8200-21idleruntime-cro21:00:000
2387799280,13cyclictest22628-21perf00:00:010
32542271,2sleep13410-21users20:45:281
2389499278,9cyclictest0-21swapper/320:35:033
2389499276,3cyclictest0-21swapper/300:27:483
2389499271,12cyclictest0-21swapper/320:04:363
2389499271,12cyclictest0-21swapper/319:37:243
2389499271,10cyclictest0-21swapper/322:35:313
23894992710,3cyclictest0-21swapper/320:07:563
23894992710,3cyclictest0-21swapper/320:07:553
2389499270,13cyclictest0-21swapper/323:58:163
2389499270,11cyclictest0-21swapper/321:51:363
2388999279,3cyclictest27377-21awk22:55:172
2388999271,12cyclictest0-21swapper/221:20:102
2388999271,12cyclictest0-21swapper/221:20:102
2388999271,10cyclictest15607-21sed23:40:262
2388999271,10cyclictest15607-21sed23:40:262
2388999271,10cyclictest0-21swapper/221:06:382
2388999271,10cyclictest0-21swapper/220:44:062
2388999270,14cyclictest0-21swapper/223:10:202
2388999270,12cyclictest0-21swapper/219:43:212
2389499269,3cyclictest0-21swapper/323:12:433
2389499269,3cyclictest0-21swapper/323:06:273
2389499269,3cyclictest0-21swapper/322:06:163
2389499268,3cyclictest0-21swapper/320:23:313
23894992612,3cyclictest0-21swapper/321:15:033
2389499261,11cyclictest0-21swapper/321:00:553
2389499261,10cyclictest0-21swapper/323:26:393
2389499260,11cyclictest0-21swapper/319:46:123
2389499260,10cyclictest0-21swapper/300:20:523
23889992615,6cyclictest0-21swapper/222:02:292
2388999261,12cyclictest0-21swapper/200:26:072
2388999261,11cyclictest0-21swapper/221:45:162
2388999261,10cyclictest0-21swapper/222:10:212
2388999261,10cyclictest0-21swapper/220:22:392
2388999260,17cyclictest0-21swapper/221:57:522
23884992619,4cyclictest4019-21/usr/sbin/munin20:50:141
2388499260,15cyclictest22399-21cstates22:45:121
40772252,3sleep14169-21/usr/sbin/munin19:35:161
302352250,2sleep30-21swapper/323:00:243
2389499258,4cyclictest0-21swapper/321:13:233
2389499257,3cyclictest0-21swapper/321:31:403
2389499257,10cyclictest0-21swapper/300:07:363
23894992511,3cyclictest0-21swapper/322:02:203
2389499250,9cyclictest0-21swapper/323:22:233
2388999251,11cyclictest0-21swapper/219:15:342
2388999250,12cyclictest0-21swapper/220:16:292
2388499256,10cyclictest9089-21cat19:45:221
2388499251,6cyclictest22947-21timerandwakeup21:30:221
2388499251,18cyclictest0-21swapper/122:53:591
2387799251,3cyclictest0-21swapper/021:40:010
2389499249,3cyclictest0-21swapper/322:47:113
2389499249,3cyclictest0-21swapper/322:24:593
2389499248,4cyclictest0-21swapper/322:40:113
2389499248,3cyclictest0-21swapper/321:09:393
2389499248,3cyclictest0-21swapper/320:31:033
2389499247,10cyclictest0-21swapper/300:33:043
23894992410,3cyclictest0-21swapper/320:18:073
2388999249,3cyclictest29496-21sh23:00:152
2388999243,8cyclictest0-21swapper/223:20:182
2388999241,10cyclictest4361-21switchtime23:15:262
2388999241,10cyclictest0-21swapper/219:35:152
2388999241,10cyclictest0-21swapper/219:32:432
2388999240,9cyclictest0-21swapper/222:08:412
2388999240,9cyclictest0-21swapper/221:29:522
2388999240,11cyclictest0-21swapper/223:30:242
2388999240,10cyclictest0-21swapper/223:05:052
2388999240,10cyclictest0-21swapper/200:00:222
2388499244,11cyclictest87550irq/128-enp1s0-00:20:141
2388499241,12cyclictest6861-21smart_sda19:40:221
2387799248,14cyclictest0-21swapper/020:50:010
123982240,3sleep012400-21proc_pri22:20:210
2389499239,3cyclictest0-21swapper/322:10:323
23894992310,3cyclictest0-21swapper/323:18:153
2388999234,3cyclictest0-21swapper/200:18:032
2388999231,9cyclictest0-21swapper/223:50:282
2388999231,9cyclictest0-21swapper/220:31:562
2388999231,19cyclictest0-21swapper/221:51:202
2388999231,11cyclictest0-21swapper/219:58:252
2388999230,10cyclictest0-21swapper/222:41:472
2388499238,3cyclictest0-21swapper/122:06:231
23884992316,4cyclictest30969-21fschecks_time21:50:141
2388499231,19cyclictest11153-21latency_hist22:20:011
2388499231,11cyclictest18279-21ntp_states21:20:211
2388499231,11cyclictest18279-21ntp_states21:20:201
2388499231,11cyclictest0-21swapper/123:37:331
2388499231,11cyclictest0-21swapper/123:04:001
2388499231,10cyclictest0-21swapper/100:25:061
132632230,2sleep00-21swapper/019:55:220
2388999227,3cyclictest27360-21grep20:30:022
2388999227,3cyclictest0-21swapper/220:48:062
2388999221,10cyclictest0-21swapper/221:02:182
23889992210,3cyclictest0-21swapper/219:28:102
2388999220,9cyclictest0-21swapper/221:43:542
2388999220,9cyclictest0-21swapper/220:07:412
2388999220,9cyclictest0-21swapper/220:07:412
2388999220,9cyclictest0-21swapper/219:12:092
2388999220,8cyclictest0-21swapper/221:12:122
2388999220,10cyclictest0-21swapper/223:28:082
2388999220,10cyclictest0-21swapper/222:20:112
2388499227,9cyclictest0-21swapper/121:13:081
2388499227,4cyclictest0-21swapper/122:12:321
2388499222,19cyclictest0-21swapper/123:27:511
23884992212,2cyclictest30433-21smart_sda19:20:221
2388499221,11cyclictest22952-21sh20:20:011
2388499221,11cyclictest0-21swapper/122:04:421
2388499221,10cyclictest0-21swapper/123:53:321
2388499221,10cyclictest0-21swapper/123:46:071
2388499221,10cyclictest0-21swapper/100:10:091
2387799220,5cyclictest21212-21kworker/u8:222:55:290
2388999216,2cyclictest0-21swapper/222:36:092
2388999215,3cyclictest0-21swapper/222:28:152
2388999211,7cyclictest0-21swapper/223:45:562
2388999211,13cyclictest0-21swapper/200:10:182
2388999211,10cyclictest26126-21ntp_states00:05:202
2388999210,7cyclictest0-21swapper/219:20:202
2388999210,10cyclictest0-21swapper/221:32:442
2388499219,4cyclictest10340-21diskstats19:50:111
2388499218,6cyclictest25654-21kworker/1:121:45:521
2388499217,3cyclictest257192expr20:25:131
2388499214,4cyclictest0-21swapper/123:43:231
2388499214,4cyclictest0-21swapper/123:43:231
2388499212,11cyclictest23527-21df_inode20:20:131
2388499212,10cyclictest822-21/usr/sbin/munin23:10:101
2388499211,9cyclictest0-21swapper/100:15:201
2388499211,17cyclictest3717-21head23:15:171
2388499211,17cyclictest24984-21ps21:35:231
2388499211,17cyclictest22699-21cut00:00:011
2388499211,15cyclictest0-21swapper/119:25:391
2388499211,11cyclictest31044-21cpu23:05:131
2388499211,11cyclictest21543-21cat21:30:021
2388499210,3cyclictest15735-21basename21:15:171
2388499210,18cyclictest12715-21cut19:55:141
2387799215,3cyclictest87350irq/126-enp1s0-20:30:230
2387799211,6cyclictest0-21swapper/021:24:180
2387799211,6cyclictest0-21swapper/021:24:180
23877992111,3cyclictest21998-21awk22:45:020
2387799211,12cyclictest14048-21/usr/sbin/munin23:40:130
2387799211,12cyclictest14048-21/usr/sbin/munin23:40:130
2387799211,10cyclictest25648-21gzip22:50:230
2388999204,8cyclictest0-21swapper/221:17:322
2388999201,3cyclictest0-21swapper/222:15:342
2388999201,12cyclictest0-21swapper/222:54:012
2388999201,11cyclictest19370-21perl20:10:142
2388499202,16cyclictest0-21swapper/122:27:401
2388499202,16cyclictest0-21swapper/121:00:051
2388499202,16cyclictest0-21swapper/120:56:121
2388499202,16cyclictest0-21swapper/120:35:391
2388499201,9cyclictest25407-21df_inode00:05:121
2388499201,8cyclictest0-21swapper/100:30:181
23884992013,3cyclictest0-21swapper/119:17:331
23884992012,3cyclictest0-21swapper/119:31:291
2388499201,18cyclictest0-21swapper/120:13:361
2388499201,16cyclictest0-21swapper/122:55:171
2388499201,14cyclictest0-21swapper/119:13:231
2388499201,11cyclictest23217-21fschecks_count00:00:141
2388499201,11cyclictest0-21swapper/122:44:181
2388499201,10cyclictest14987-21grep20:00:141
2388499201,10cyclictest1145-21unixbench_singl20:40:251
2388499201,10cyclictest10840-21expr21:05:131
2388499201,10cyclictest0-21swapper/122:34:061
2388499201,10cyclictest0-21swapper/122:34:061
2388499201,10cyclictest0-21swapper/121:59:561
2388499201,10cyclictest0-21swapper/100:39:011
2387799201,5cyclictest0-21swapper/023:15:480
2387799201,5cyclictest0-21swapper/022:25:160
23877992014,3cyclictest1493-21gltestperf23:10:130
2387799201,11cyclictest31027-21/usr/sbin/munin23:05:230
2387799201,11cyclictest22217-21/usr/sbin/munin22:45:190
2387799201,11cyclictest16600-21/usr/sbin/munin20:05:250
2387799201,11cyclictest16600-21/usr/sbin/munin20:05:250
2388999197,3cyclictest0-21swapper/200:32:232
2388999191,9cyclictest0-21swapper/219:54:272
2388499197,3cyclictest0-21swapper/123:24:381
2388499192,14cyclictest13210-21munin-run22:25:001
2388499191,5cyclictest19412-21users22:35:261
2388499190,13cyclictest0-21swapper/123:30:171
2387799199,3cyclictest32598-21/usr/sbin/munin21:55:180
2387799191,16cyclictest9429-21munin-run23:30:000
2387799191,10cyclictest29468-21/usr/sbin/munin00:15:120
2387799190,10cyclictest7439-21gltestperf22:10:150
211082191,3sleep00-21swapper/021:26:430
2388499181,16cyclictest0-21swapper/121:41:201
2388499181,10cyclictest0-21swapper/120:05:191
2388499181,10cyclictest0-21swapper/120:05:181
2387799184,4cyclictest23114-21sed21:30:230
2387799183,3cyclictest21212-21kworker/u8:222:34:320
2387799183,3cyclictest21212-21kworker/u8:222:34:320
2387799181,8cyclictest0-21swapper/000:27:490
2387799181,4cyclictest4275-21fschecks_time00:30:160
23877991811,4cyclictest23482-21gltestperf00:00:150
2387799181,11cyclictest0-21swapper/023:50:240
2387799181,10cyclictest0-21swapper/021:50:490
136872181,2sleep00-21swapper/023:38:270
2387799173,8cyclictest23867-21cyclictest20:50:140
2387799173,7cyclictest10665-21cpu21:05:120
2387799173,3cyclictest6196-21sed22:05:250
2387799173,3cyclictest28808-21head21:45:140
2387799171,8cyclictest20757-21idleruntime-cro20:15:010
2387799171,6cyclictest0-21swapper/023:04:000
2387799171,3cyclictest0-21swapper/000:20:170
2387799171,2cyclictest0-21swapper/022:39:370
2387799171,2cyclictest0-21swapper/021:15:170
23877991712,3cyclictest32662-21hwlatdetect20:40:140
2387799171,15cyclictest0-21swapper/019:31:070
2387799171,15cyclictest0-21swapper/019:11:010
23877991711,4cyclictest9797-21cut23:30:120
2387799171,14cyclictest970-21perf19:30:010
2387799171,14cyclictest0-21swapper/019:50:040
23877991711,3cyclictest808-21snmpd19:49:250
2387799171,10cyclictest0-21swapper/019:35:200
180952171,2sleep00-21swapper/023:48:340
267092160,2sleep00-21swapper/000:06:000
2387799162,12cyclictest29290-21cut19:20:110
2387799161,8cyclictest21064-21cat20:15:130
2387799161,8cyclictest15852-21unixbench-2d20:00:250
2387799161,6cyclictest0-21swapper/021:40:110
2387799161,4cyclictest9422-21sa121:00:200
2387799161,3cyclictest0-21swapper/021:10:150
2387799161,2cyclictest0-21swapper/020:25:170
2387799161,2cyclictest0-21swapper/019:40:120
23877991611,3cyclictest7149-21sendmail_mailtr00:35:230
23877991610,3cyclictest29950-21grep20:35:120
23877991610,3cyclictest2522-21cpuspeed_turbos22:00:130
2387799160,6cyclictest0-21swapper/023:20:250
2387799160,3cyclictest0-21swapper/020:20:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional