You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-04 - 16:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sat Apr 04, 2026 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2305328996453,8cyclictest2508982-21latency_hist12:35:021
2305329996054,3cyclictest2336893-21kworker/u8:10-xprtiod09:37:042
230532999541,24cyclictest2432878-21ssh11:00:342
2305329995128,6cyclictest2353617-21munin-plugin-st09:25:012
2305329995020,4cyclictest0-21swapper/208:40:352
2305329994925,4cyclictest2417280-21kworker/u8:6-rpciod11:10:262
2305327994841,5cyclictest2432435-21sshd11:00:250
2305329994738,6cyclictest421-21dbus-daemon07:35:022
2305328994731,5cyclictest171rcu_preempt12:40:011
2305328994640,3cyclictest0-21swapper/107:40:011
230532799468,28cyclictest2393006-21ssh10:11:250
230532999456,30cyclictest2498054-21ssh12:20:362
230532999452,37cyclictest2379518-21ssh09:55:312
2305329994434,6cyclictest2473933-21timerandwakeup11:50:422
2305328994437,3cyclictest0-21swapper/111:40:011
230532799449,33cyclictest2431274-21ssh10:59:210
230532799447,35cyclictest2373126-21ssh09:47:560
2305329994335,5cyclictest2423469-21cron10:50:002
2305329994335,5cyclictest2349501-21munin-plugin-st09:20:012
2305329994335,5cyclictest2308321-21idleruntime-cro07:20:002
2305329994334,5cyclictest2314303-21munin-run07:40:012
2305328994335,5cyclictest2443880-21systemd-run11:15:021
2305328994335,5cyclictest2419420-21systemd-run10:45:011
2305328994332,8cyclictest2346830-21mailstats09:15:401
230532799437,34cyclictest2467428-21ssh11:43:200
230532799435,28cyclictest2384755-21ssh10:01:260
230532799433,35cyclictest2310711-21ntpq07:25:320
2305327994320,20cyclictest2510726-21unixbench_singl12:35:420
230532799430,6cyclictest2502748-21ssh12:25:550
2305329994235,3cyclictest0-21swapper/212:00:292
2305329994234,5cyclictest2428199-21cpuspeed10:55:232
230532999422,32cyclictest2477803-21ssh11:55:372
230532799427,33cyclictest2456334-21ssh11:30:040
230532799427,33cyclictest2443463-21ssh11:14:180
230532799422,35cyclictest2489735-21ntpq12:10:290
230532799422,35cyclictest2489735-21ntpq12:10:290
230532799421,6cyclictest2462971-21sh11:37:380
230532799420,6cyclictest2426833-21ssh10:53:390
2305329994132,6cyclictest2447914-21awk11:20:012
2305329994132,6cyclictest2447914-21awk11:20:012
2305329994131,7cyclictest2327818-21idleruntime-cro08:25:012
2305329994131,5cyclictest0-21swapper/209:35:012
230532899415,30cyclictest2465405-21ssh11:40:311
2305328994132,6cyclictest2480500-21sh12:00:011
2305328994131,7cyclictest2364364-21sh09:37:041
230532899412,33cyclictest2341295-21diskmemload09:40:301
230532899412,33cyclictest2341295-21diskmemload09:40:301
230532799413,3cyclictest2398921-21ssh10:19:510
230532799413,33cyclictest2318281-21ntpq07:50:360
230532799413,33cyclictest2306166-21ntpq07:10:290
230532799411,5cyclictest2334661-21ntpq08:45:290
230532799411,4cyclictest2485819-21ntpq12:05:350
230532799410,6cyclictest2401101-21ssh10:21:120
230532799410,3cyclictest2413063-21ssh10:35:440
2305329994031,6cyclictest420-21avahi-daemon11:25:492
2305329994030,7cyclictest2367716-21grep09:40:432
2305329994030,7cyclictest2367716-21grep09:40:432
2305329994029,8cyclictest2330170-21ntp_kernel_pll_08:30:302
2305328994033,3cyclictest0-21swapper/112:10:301
2305328994033,3cyclictest0-21swapper/112:10:301
230532899403,29cyclictest2436894-21ssh11:05:321
2305328994023,14cyclictest2424628-21ssh10:50:321
230532799405,24cyclictest0-21swapper/012:17:560
230532799402,7cyclictest2473537-21ntpq11:50:320
230532799402,6cyclictest2497946-21ntpq12:20:320
230532799402,5cyclictest2315218-21ntpq07:40:310
230532799402,34cyclictest2375344-21ntpq09:50:300
230532799402,33cyclictest2324182-21ntpq08:10:310
2305327994023,14cyclictest1430195-21cups-browsed10:25:410
2305327994023,14cyclictest1430195-21cups-browsed10:25:410
2305327994022,15cyclictest0-21swapper/011:45:320
230532799401,6cyclictest2387932-21ntpq10:05:360
230532799401,5cyclictest2354915-21ntpq09:25:380
230532799401,5cyclictest2339228-21ntpq09:00:310
230532799400,6cyclictest2478288-21ssh11:55:470
2305329993934,2cyclictest0-21swapper/212:10:012
2305329993932,3cyclictest0-21swapper/212:35:392
2305329993930,6cyclictest2326319-21sendmail08:20:012
2305329993930,6cyclictest1430177-21cupsd11:24:432
2305329993930,6cyclictest1430177-21cupsd11:24:432
2305329993927,9cyclictest2469386-21ntp_kernel_err11:45:282
2305328993930,3cyclictest0-21swapper/110:10:311
230532799392,5cyclictest2328717-21ntpq08:25:310
230532799392,3cyclictest2322683-21ntpq08:05:300
230532799392,32cyclictest2342469-21ssh09:10:320
230532799392,32cyclictest2319780-21ntpq07:55:360
230532799391,5cyclictest2316695-21ntpq07:45:310
230532799391,4cyclictest2337761-21ntpq08:55:350
230532799391,4cyclictest2337761-21ntpq08:55:350
230532799391,3cyclictest2379659-21ntpq09:55:360
230532799391,35cyclictest2422033-21ssh10:47:180
230532799390,4cyclictest2418894-21ssh10:44:030
230532799390,37cyclictest2438439-21ssh11:07:270
230532799390,37cyclictest2351805-21ssh09:21:350
230532999383,27cyclictest2354896-21ssh09:25:372
2305329993832,3cyclictest0-21swapper/208:30:012
2305329993831,4cyclictest0-21swapper/209:15:002
2305329993830,5cyclictest2411231-21munin-plugin-st10:35:002
2305329993830,5cyclictest2411231-21munin-plugin-st10:35:002
2305329993830,4cyclictest2464247-21sendmail11:40:012
2305329993823,9cyclictest2502080-21ssh12:25:362
230532799382,5cyclictest2321194-21ntpq08:00:320
230532799382,32cyclictest2327301-21ntpq08:20:360
2305327993823,12cyclictest420-21avahi-daemon09:42:240
2305327993823,12cyclictest420-21avahi-daemon09:42:240
230532799381,9cyclictest2480872-21ssh12:00:160
230532799381,5cyclictest2449083-21ntpq11:20:310
230532799381,5cyclictest2449083-21ntpq11:20:310
230532799381,5cyclictest2325678-21ntpq08:15:310
230532799381,4cyclictest2313750-21ntpq07:35:330
230532799381,3cyclictest2453123-21ntpq11:25:310
230532799381,3cyclictest2357958-21ssh09:30:040
230532799381,3cyclictest2331636-21ntpq08:35:280
230532799381,12cyclictest2444968-21ntpq11:15:310
230532799381,12cyclictest2444968-21ntpq11:15:310
230532799381,12cyclictest2312275-21ntpq07:30:340
230532799380,9cyclictest2363883-21ssh09:36:100
230532799380,5cyclictest2348350-21ssh09:17:520
2305329993729,5cyclictest2508947-21idleruntime-cro12:35:012
2305328993729,5cyclictest2460260-21ssh11:35:021
2305328993729,5cyclictest2354337-21fschecks_count09:25:251
2305328993711,15cyclictest2379521-21ntpq09:55:311
230532799372,5cyclictest2307652-21ntpq07:15:300
230532799372,4cyclictest2410816-21ssh10:34:150
230532799372,4cyclictest2410816-21ssh10:34:150
230532799371,5cyclictest2505954-21ntpq12:30:290
230532799371,4cyclictest2340747-21ntpq09:05:320
230532799371,4cyclictest2340747-21ntpq09:05:320
230532799371,4cyclictest2333196-21ntpq08:40:320
230532799371,4cyclictest2309189-21ntpq07:20:300
230532799371,3cyclictest2336263-21ntpq08:50:360
230532799371,3cyclictest2336263-21ntpq08:50:360
2305329993629,5cyclictest2305326-21cyclictest09:55:002
2305329993628,5cyclictest2332299-21systemctl08:40:012
2305329993627,6cyclictest516-21in:imuxsock07:25:002
2305329993627,6cyclictest421-21dbus-daemon10:37:252
2305328993629,3cyclictest0-21swapper/111:00:291
2305328993628,5cyclictest2500846-21sh12:25:011
2305328993628,5cyclictest2428134-21aten_r2power_vo10:55:221
2305328993626,7cyclictest2400260-21ssh10:20:341
2305328993626,6cyclictest2383872-21ssh10:00:371
230532799361,3cyclictest2330181-21ntpq08:30:300
230532999354,27cyclictest2387916-21ssh10:05:352
2305329993527,3cyclictest0-21swapper/210:20:352
2305329993526,4cyclictest0-21swapper/209:10:022
2305329993526,4cyclictest0-21swapper/209:10:022
2305329993524,8cyclictest420-21avahi-daemon10:43:502
2305329993524,8cyclictest0-21swapper/210:10:242
230532999352,25cyclictest2436855-21ssh11:05:312
230532999352,23cyclictest2396132-21ssh10:15:342
230532899355,19cyclictest2501946-21ssh12:25:321
2305328993528,3cyclictest2370360-21kworker/u8:5-xprtiod10:15:431
2305328993527,4cyclictest0-21swapper/110:30:351
2305328993527,4cyclictest0-21swapper/110:30:351
2305328993526,5cyclictest2321832-21systemd-run08:05:011
2305328993525,6cyclictest1430195-21cups-browsed12:08:001
2305328993524,7cyclictest1430195-21cups-browsed11:20:301
2305328993524,7cyclictest1430195-21cups-browsed11:20:301
2305328993511,8cyclictest2455569-21sh11:29:031
230532999341,25cyclictest2383686-21sh10:00:312
2305328993426,5cyclictest2351070-21timerandwakeup09:20:401
2305328993422,9cyclictest2342936-21timerwakeupswit09:10:421
2305329993325,4cyclictest0-21swapper/209:00:002
2305329993325,4cyclictest0-21swapper/209:00:002
2305329993325,3cyclictest0-21swapper/210:25:292
2305329993325,3cyclictest0-21swapper/210:25:292
2305329993324,4cyclictest0-21swapper/211:40:292
2305329993321,9cyclictest421-21dbus-daemon12:15:022
2305329993321,9cyclictest421-21dbus-daemon12:15:022
2305329993317,11cyclictest2371726-21/usr/sbin/munin09:45:412
2305328993326,5cyclictest0-21swapper/109:50:001
2305328993326,4cyclictest0-21swapper/109:55:001
2305328993325,4cyclictest2435779-21kworker/u8:11-events_unbound12:15:211
2305328993324,6cyclictest548-21gdbus07:25:011
2305328993311,8cyclictest2361187-21sh09:33:511
230532799331,4cyclictest2305339-21munin-run07:10:010
230532999323,23cyclictest2456976-21ssh11:30:252
230532999323,19cyclictest428-21mta-sts-daemon07:45:322
2305329993225,3cyclictest0-21swapper/212:20:002
2305328993226,3cyclictest0-21swapper/110:05:341
2305328993225,3cyclictest0-21swapper/112:00:361
2305328993224,5cyclictest2419861-21aten_r2power_en10:45:201
2305328993224,5cyclictest2336778-21cron08:55:001
2305328993224,5cyclictest2336778-21cron08:55:001
2305328993218,11cyclictest2325761-21ntpq08:15:351
2305328993210,7cyclictest2406084-21sh10:27:561
2305328993210,7cyclictest2406084-21sh10:27:561
230532999313,23cyclictest659-21snmpd08:45:352
2305329993124,3cyclictest0-21swapper/210:55:012
2305328993124,3cyclictest0-21swapper/110:35:241
2305328993121,7cyclictest2444831-21missed_timers11:15:271
2305328993121,7cyclictest2444831-21missed_timers11:15:271
2305328993118,8cyclictest1055-21mta-sts-daemon09:05:281
2305328993118,8cyclictest1055-21mta-sts-daemon09:05:281
2305328993114,11cyclictest2334023-21/usr/sbin/munin08:45:431
2305328993110,13cyclictest0-21swapper/107:45:301
230532899307,20cyclictest261-21systemd-journal07:10:561
2305328993025,2cyclictest0-21swapper/109:05:011
2305328993023,4cyclictest1054-21mta-sts-daemon07:20:001
2305328993023,3cyclictest0-21swapper/111:45:411
2305328993022,5cyclictest600-21gmain11:51:321
2305328993021,6cyclictest420-21avahi-daemon08:30:011
2305328993016,11cyclictest2315381-21sendmail_mailst07:40:371
2305328993016,10cyclictest2319659-21ntpq07:55:301
2305329992921,5cyclictest2320290-21latency_hist08:00:012
2305329992921,5cyclictest1430195-21cups-browsed08:06:492
2305328992914,11cyclictest2310703-21ntpq07:25:321
2305328992913,12cyclictest2327188-21ntpq08:20:301
2305328992822,3cyclictest1051-21mta-sts-daemon08:40:021
2305328992822,3cyclictest0-21swapper/109:00:001
2305328992822,3cyclictest0-21swapper/109:00:001
2305328992816,6cyclictest2330277-21ntpq08:30:351
2305329992719,5cyclictest2315389-21systemctl07:40:372
2305328992721,3cyclictest0-21swapper/108:15:011
2305328992719,5cyclictest2317841-21fschecks_time07:50:251
2305329992620,3cyclictest0-21swapper/207:55:012
2305329992619,4cyclictest439-21systemd-logind07:15:012
2305329992617,7cyclictest2336591-21timerwakeupswit08:50:422
2305329992617,7cyclictest2336591-21timerwakeupswit08:50:422
2305329992616,8cyclictest2324794-21idleruntime-cro08:15:002
2305328992617,7cyclictest2322323-21fschecks_count08:05:241
2305328992613,7cyclictest2312229-21ntpq07:30:321
2305328992610,13cyclictest2333138-21netstat08:40:291
2305329992417,3cyclictest0-21swapper/208:00:422
2305329992313,7cyclictest2339803-21debian-sa109:05:002
2305329992217,2cyclictest0-21swapper/207:30:002
2305328992212,6cyclictest428-21mta-sts-daemon07:10:011
230532999215,11cyclictest421-21dbus-daemon07:10:012
230533099193,10cyclictest0-21swapper/309:37:033
230533099181,8cyclictest0-21swapper/309:15:323
230533099163,6cyclictest0-21swapper/311:05:353
230533099161,8cyclictest0-21swapper/311:20:273
230533099161,8cyclictest0-21swapper/311:20:273
230533099161,6cyclictest0-21swapper/309:00:283
230533099152,6cyclictest0-21swapper/309:30:323
230533099151,9cyclictest0-21swapper/312:05:423
230533099151,8cyclictest0-21swapper/312:00:313
230533099151,6cyclictest0-21swapper/307:20:333
230533099142,8cyclictest0-21swapper/311:45:343
230533099141,7cyclictest0-21swapper/311:50:263
230533099141,7cyclictest0-21swapper/311:30:303
230533099141,7cyclictest0-21swapper/310:55:333
230533099141,7cyclictest0-21swapper/309:40:383
230533099141,7cyclictest0-21swapper/309:40:383
230533099141,7cyclictest0-21swapper/307:40:303
230533099141,6cyclictest0-21swapper/309:05:283
230533099141,6cyclictest0-21swapper/309:05:283
230533099141,6cyclictest0-21swapper/308:10:353
230533099141,6cyclictest0-21swapper/308:00:333
230533099133,7cyclictest0-21swapper/310:45:303
230533099133,4cyclictest0-21swapper/307:50:343
230533099132,7cyclictest0-21swapper/311:55:323
230533099132,7cyclictest0-21swapper/311:20:013
230533099132,7cyclictest0-21swapper/311:20:013
230533099132,7cyclictest0-21swapper/309:20:253
230533099132,7cyclictest0-21swapper/308:30:233
230533099132,6cyclictest0-21swapper/312:30:313
230533099132,6cyclictest0-21swapper/307:30:293
230533099131,8cyclictest0-21swapper/310:52:243
230533099131,8cyclictest0-21swapper/310:05:343
230533099131,7cyclictest0-21swapper/312:25:253
230533099131,7cyclictest0-21swapper/312:10:243
230533099131,7cyclictest0-21swapper/312:10:243
230533099131,7cyclictest0-21swapper/311:40:323
230533099131,7cyclictest0-21swapper/311:40:013
230533099131,7cyclictest0-21swapper/310:40:023
230533099131,7cyclictest0-21swapper/310:30:263
230533099131,7cyclictest0-21swapper/310:30:263
230533099131,7cyclictest0-21swapper/309:55:333
230533099131,6cyclictest0-21swapper/312:25:023
230533099131,6cyclictest0-21swapper/308:25:283
230533099131,5cyclictest0-21swapper/307:45:293
230533099123,6cyclictest0-21swapper/309:30:013
230533099123,6cyclictest0-21swapper/309:10:263
230533099122,6cyclictest0-21swapper/311:10:393
230533099121,7cyclictest0-21swapper/312:35:303
230533099121,7cyclictest0-21swapper/312:15:313
230533099121,7cyclictest0-21swapper/311:04:553
230533099121,7cyclictest0-21swapper/310:00:323
230533099121,7cyclictest0-21swapper/309:45:323
230533099121,7cyclictest0-21swapper/308:35:293
230533099121,6cyclictest0-21swapper/307:55:363
230533099121,6cyclictest0-21swapper/307:30:013
230533099121,5cyclictest0-21swapper/311:30:023
230533099121,5cyclictest0-21swapper/307:35:313
230533099121,5cyclictest0-21swapper/307:15:313
230533099120,7cyclictest0-21swapper/310:20:213
230533099113,6cyclictest0-21swapper/309:50:233
230533099111,7cyclictest0-21swapper/310:15:253
230533099111,7cyclictest0-21swapper/310:10:293
230533099111,6cyclictest0-21swapper/308:55:013
230533099111,6cyclictest0-21swapper/308:55:013
230533099111,5cyclictest0-21swapper/310:45:023
230533099110,5cyclictest0-21swapper/308:15:293
230533099103,4cyclictest0-21swapper/308:40:313
230533099101,6cyclictest0-21swapper/310:25:193
230533099101,6cyclictest0-21swapper/310:25:193
230533099101,5cyclictest0-21swapper/308:45:253
230533099101,5cyclictest0-21swapper/308:05:213
230533099101,5cyclictest0-21swapper/307:10:263
23053309991,5cyclictest0-21swapper/308:55:233
23053309991,5cyclictest0-21swapper/308:55:233
23053309991,4cyclictest0-21swapper/307:10:023
23053309981,5cyclictest0-21swapper/308:20:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional