You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-10 - 19:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Wed Dec 10, 2025 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30204993129,1cyclictest28-21ksoftirqd/211:15:002
3020399263,22cyclictest24383-21idleruntime-cro09:10:001
3020399262,23cyclictest14050-21basename09:55:001
3020399262,23cyclictest14050-21basename09:54:591
3020599240,23cyclictest313-21systemd-journal07:35:003
3020499240,23cyclictest3349-21grep10:40:002
3020499240,0cyclictest0-21swapper/212:18:462
3020299240,23cyclictest29500-21sendmail-msp12:40:000
3020599230,22cyclictest14930-21apt-get08:53:443
30204992321,1cyclictest9544-21sendmail08:40:002
30204992320,2cyclictest903-21dump-pmu-power10:35:002
3020499230,0cyclictest0-21swapper/208:44:592
3020399230,22cyclictest612-21gdbus09:40:001
3020399230,22cyclictest24959-21perf11:25:001
3020399230,0cyclictest20697-21apt-get11:18:481
3020299230,22cyclictest17886-21dump-pmu-power07:50:000
30205992220,1cyclictest25117-21grep08:05:003
30205992220,1cyclictest12620-21perf12:05:003
30205992220,1cyclictest11993-21perf08:45:003
30205992220,1cyclictest0-21swapper/308:40:003
3020599220,22cyclictest0-21swapper/309:33:463
3020599220,1cyclictest0-21swapper/307:45:023
30204992221,1cyclictest0-21swapper/211:53:462
30204992220,1cyclictest3171-21apt-get11:48:462
30204992219,2cyclictest22568-21munin-run11:20:002
3020499220,21cyclictest6420-21cron09:40:002
3020499220,21cyclictest3895-21apt-get10:43:462
3020499220,1cyclictest21266-21perf10:10:012
3020499220,0cyclictest0-21swapper/207:48:442
30203992220,1cyclictest0-21swapper/110:25:001
30203992219,2cyclictest612-21gdbus12:20:001
3020399220,21cyclictest15113-21apt-get07:45:021
3020399220,21cyclictest0-21swapper/109:43:441
3020399220,21cyclictest0-21swapper/108:13:461
30202992220,1cyclictest22716-21perf08:00:000
30202992220,1cyclictest0-21swapper/007:23:460
3020299220,21cyclictest9812-21apt-get09:48:450
3020299220,21cyclictest0-21swapper/010:53:450
3020299220,21cyclictest0-21swapper/007:33:450
3020299220,21cyclictest0-21swapper/007:13:430
3020299220,1cyclictest15028-21awk12:10:000
30205992121,0cyclictest0-21swapper/311:18:483
30205992121,0cyclictest0-21swapper/309:48:463
30205992121,0cyclictest0-21swapper/309:23:453
30205992121,0cyclictest0-21swapper/308:23:453
3020599210,21cyclictest311-21apt-get11:43:443
3020599210,20cyclictest0-21swapper/311:48:463
3020599210,1cyclictest32596-21perf07:15:003
3020599210,0cyclictest0-21swapper/311:53:463
3020599210,0cyclictest0-21swapper/309:55:003
3020599210,0cyclictest0-21swapper/309:55:003
30204992120,0cyclictest0-21swapper/208:33:452
30204992119,1cyclictest23656-21cron10:15:002
3020499210,20cyclictest0-21swapper/211:58:462
3020499210,20cyclictest0-21swapper/209:48:462
3020499210,1cyclictest24937-21apt-get09:13:482
3020499210,1cyclictest17720-21cron11:10:002
3020499210,1cyclictest0-21swapper/207:53:472
3020499210,0cyclictest0-21swapper/212:33:472
3020499210,0cyclictest0-21swapper/212:08:452
30203992121,0cyclictest0-21swapper/110:18:461
30203992120,1cyclictest31485-21apt-get10:33:451
30203992120,1cyclictest13263-21apt-get07:43:461
30203992120,1cyclictest0-21swapper/111:08:451
30203992120,0cyclictest0-21swapper/109:03:451
30203992119,1cyclictest211rcuc/108:20:001
3020399210,21cyclictest0-21swapper/111:48:451
3020399210,20cyclictest0-21swapper/111:38:441
3020399210,20cyclictest0-21swapper/108:48:471
3020399210,20cyclictest0-21swapper/107:23:461
3020399210,0cyclictest0-21swapper/108:08:451
30202992120,0cyclictest19414-21apt-get10:08:450
30202992119,1cyclictest27328-21cron11:29:590
3020299210,21cyclictest0-21swapper/011:08:450
3020299210,21cyclictest0-21swapper/008:28:450
3020299210,20cyclictest8354-21apt-get11:58:460
3020299210,20cyclictest0-21swapper/009:13:470
3020299210,18cyclictest0-21swapper/010:13:450
3020299210,0cyclictest5957-21apt-get11:53:460
3020299210,0cyclictest18296-21apt-get11:13:470
3020299210,0cyclictest0-21swapper/012:03:450
3020299210,0cyclictest0-21swapper/008:38:440
3020299210,0cyclictest0-21swapper/008:13:450
30205992020,0cyclictest5548-21apt-get07:28:483
30205992020,0cyclictest24213-21apt-get10:18:463
30205992020,0cyclictest10443-21apt-get07:38:443
30205992020,0cyclictest0-21swapper/310:38:483
30205992020,0cyclictest0-21swapper/310:03:443
30205992020,0cyclictest0-21swapper/308:55:023
30205992020,0cyclictest0-21swapper/308:48:473
30205992020,0cyclictest0-21swapper/308:33:453
30205992019,1cyclictest697-21apt-get07:18:443
3020599200,0cyclictest27959-21apt-get11:33:443
3020599200,0cyclictest15897-21apt-get11:08:453
30204992020,0cyclictest0-21swapper/208:55:022
30204992020,0cyclictest0-21swapper/207:43:452
30204992020,0cyclictest0-21swapper/207:38:442
30204992020,0cyclictest0-21swapper/207:13:432
30204992019,1cyclictest27572-21apt-get12:38:462
30204992019,1cyclictest17018-21apt-get10:03:452
30204992018,1cyclictest11083-21apt-get10:58:452
30204992017,2cyclictest27371-21awk11:30:002
3020499200,20cyclictest0-21swapper/208:53:452
3020499200,20cyclictest0-21swapper/207:23:462
3020499200,19cyclictest24605-21sh12:30:002
3020499200,0cyclictest0-21swapper/209:43:442
30203992020,0cyclictest9796-21perf07:35:001
30203992020,0cyclictest0-21swapper/108:38:441
30203992019,0cyclictest5020-21apt-get11:50:011
30203992018,1cyclictest31644-21cron09:25:001
3020399200,0cyclictest0-21swapper/111:58:461
3020399200,0cyclictest0-21swapper/110:48:471
3020399200,0cyclictest0-21swapper/110:28:471
30202992020,0cyclictest0-21swapper/012:28:460
30202992020,0cyclictest0-21swapper/012:23:470
30202992020,0cyclictest0-21swapper/009:28:470
30202992020,0cyclictest0-21swapper/007:38:440
30202992019,1cyclictest18448-21apt-get07:53:460
30202992018,1cyclictest593-21dbus-daemon09:00:000
30205991919,0cyclictest0-21swapper/312:38:453
30205991919,0cyclictest0-21swapper/312:23:473
30205991919,0cyclictest0-21swapper/311:28:463
30205991919,0cyclictest0-21swapper/311:03:453
30205991919,0cyclictest0-21swapper/310:58:453
30205991919,0cyclictest0-21swapper/310:08:443
30205991919,0cyclictest0-21swapper/309:40:013
30205991919,0cyclictest0-21swapper/309:38:443
30205991919,0cyclictest0-21swapper/309:18:473
30205991918,1cyclictest13168-21apt-get12:08:453
30204991919,0cyclictest0-21swapper/211:43:442
30204991919,0cyclictest0-21swapper/211:38:452
30204991919,0cyclictest0-21swapper/208:23:452
30204991919,0cyclictest0-21swapper/208:10:002
30204991919,0cyclictest0-21swapper/207:18:452
30204991918,1cyclictest10771-21apt-get12:03:452
30204991917,1cyclictest271rcuc/209:05:002
30204991917,1cyclictest0-21swapper/212:25:002
30204991917,1cyclictest0-21swapper/209:35:002
30204991916,2cyclictest1641-21cron09:55:002
30204991916,2cyclictest1641-21cron09:55:002
3020499190,18cyclictest8685-21apt-get10:53:452
30203991919,0cyclictest15561-21apt-get12:13:451
30203991919,0cyclictest0-21swapper/111:28:461
30203991919,0cyclictest0-21swapper/110:15:001
30203991919,0cyclictest0-21swapper/109:18:471
30203991919,0cyclictest0-21swapper/109:13:471
30202991919,0cyclictest472-21apt-get08:23:450
30202991919,0cyclictest0-21swapper/011:33:450
30202991919,0cyclictest0-21swapper/011:23:460
30202991919,0cyclictest0-21swapper/010:18:460
30202991919,0cyclictest0-21swapper/009:58:430
30202991918,0cyclictest0-21swapper/008:48:470
30205991818,0cyclictest30515-21apt-get08:18:473
30205991818,0cyclictest14600-21apt-get09:58:433
30205991818,0cyclictest0-21swapper/312:13:443
30205991818,0cyclictest0-21swapper/311:13:463
30205991818,0cyclictest0-21swapper/310:53:443
30205991818,0cyclictest0-21swapper/310:43:463
30205991818,0cyclictest0-21swapper/310:13:443
30204991818,0cyclictest0-21swapper/209:58:432
30204991818,0cyclictest0-21swapper/209:08:452
30204991818,0cyclictest0-21swapper/208:28:452
30204991818,0cyclictest0-21swapper/208:18:462
30204991816,1cyclictest9778-21cron07:35:002
3020499180,17cyclictest6294-21apt-get10:48:482
30203991818,0cyclictest0-21swapper/112:38:461
30203991818,0cyclictest0-21swapper/110:58:451
30203991818,0cyclictest0-21swapper/110:43:461
30203991818,0cyclictest0-21swapper/110:03:441
30203991818,0cyclictest0-21swapper/107:28:481
30203991817,0cyclictest0-21swapper/111:03:451
30203991817,0cyclictest0-21swapper/108:55:011
30202991818,0cyclictest0-21swapper/010:38:470
30202991818,0cyclictest0-21swapper/010:28:470
30202991818,0cyclictest0-21swapper/010:23:460
30202991818,0cyclictest0-21swapper/009:23:450
30202991818,0cyclictest0-21swapper/008:43:440
30202991818,0cyclictest0-21swapper/008:18:450
30202991817,1cyclictest23263-21apt-get08:03:470
30202991817,1cyclictest16786-21perf08:55:000
30202991816,1cyclictest4025-21cron09:35:000
30205991717,0cyclictest32213-21apt-get09:28:473
30205991714,2cyclictest28533-21cron10:25:003
30204991717,0cyclictest0-21swapper/207:28:472
30204991715,1cyclictest31632-21cron09:25:002
30204991715,1cyclictest30946-21munin-run10:30:002
30204991715,1cyclictest26632-21apt-get10:23:462
30203991717,0cyclictest0-21swapper/111:33:451
30203991717,0cyclictest0-21swapper/109:28:471
30203991717,0cyclictest0-21swapper/107:53:461
30203991717,0cyclictest0-21swapper/107:38:431
30203991717,0cyclictest0-21swapper/107:13:431
30203991716,0cyclictest0-21swapper/110:00:001
3020399170,16cyclictest15093-21grep08:53:461
30202991717,0cyclictest0-21swapper/010:48:480
30202991717,0cyclictest0-21swapper/010:03:440
3020299170,1cyclictest25659-21apt-get08:08:460
3020299170,17cyclictest0-21swapper/007:43:450
30205991616,0cyclictest22777-21apt-get12:28:463
30205991616,0cyclictest0-21swapper/311:58:463
30205991616,0cyclictest0-21swapper/309:13:483
30205991616,0cyclictest0-21swapper/308:08:453
30205991616,0cyclictest0-21swapper/307:53:473
30205991615,1cyclictest20841-21apt-get07:58:443
30205991614,1cyclictest4196-21ntp_states07:23:523
3020599160,15cyclictest0-21swapper/307:40:023
30204991616,0cyclictest0-21swapper/211:23:462
30204991614,1cyclictest26090-21munin-run10:20:002
3020499160,15cyclictest15364-21basename11:04:592
30203991616,0cyclictest2203-21apt-get09:33:461
30203991616,0cyclictest0-21swapper/108:28:451
30203991616,0cyclictest0-21swapper/108:03:471
30203991614,1cyclictest0-21swapper/112:33:521
30202991616,0cyclictest0-21swapper/010:33:440
30202991616,0cyclictest0-21swapper/009:18:460
30202991615,1cyclictest13501-21apt-get11:03:460
30202991614,1cyclictest5742-21df_inode08:33:460
30202991614,1cyclictest4595-21apt-get09:38:450
3020299160,15cyclictest0-21swapper/011:18:480
30205991515,0cyclictest0-21swapper/308:13:453
30204991515,0cyclictest0-21swapper/212:13:452
30204991515,0cyclictest0-21swapper/211:35:002
30204991514,1cyclictest12534-21apt-get08:48:472
30204991514,1cyclictest0-21swapper/209:18:482
30204991513,1cyclictest0-21swapper/208:05:002
30203991515,0cyclictest0-21swapper/112:23:471
30203991515,0cyclictest0-21swapper/111:43:441
30203991515,0cyclictest0-21swapper/109:48:461
30203991515,0cyclictest0-21swapper/108:43:451
3020399150,15cyclictest0-21swapper/107:18:451
30202991515,0cyclictest0-21swapper/012:13:440
30202991515,0cyclictest0-21swapper/011:43:440
30202991515,0cyclictest0-21swapper/009:03:440
30202991514,1cyclictest0-21swapper/011:48:460
30205991414,0cyclictest0-21swapper/312:15:053
30205991414,0cyclictest0-21swapper/310:31:053
30205991414,0cyclictest0-21swapper/310:25:423
3020599140,14cyclictest0-21swapper/312:33:463
3020599140,14cyclictest0-21swapper/310:46:083
3020599140,14cyclictest0-21swapper/309:05:113
3020599140,14cyclictest0-21swapper/309:01:133
3020599140,14cyclictest0-21swapper/308:25:393
3020599140,0cyclictest0-21swapper/311:36:143
3020499140,14cyclictest0-21swapper/208:12:402
3020499140,14cyclictest0-21swapper/207:56:492
30203991414,0cyclictest0-21swapper/112:05:431
3020399140,14cyclictest0-21swapper/107:57:251
3020399140,0cyclictest0-21swapper/111:11:011
3020399140,0cyclictest0-21swapper/108:30:241
3020399140,0cyclictest0-21swapper/108:21:111
30202991414,0cyclictest0-21swapper/012:15:080
30202991414,0cyclictest0-21swapper/011:35:580
30202991414,0cyclictest0-21swapper/010:57:060
30202991413,1cyclictest6445-21apt-get09:40:010
3020299140,14cyclictest0-21swapper/009:53:120
3020299140,14cyclictest0-21swapper/009:53:120
3020299140,14cyclictest0-21swapper/007:25:350
3020299140,0cyclictest0-21swapper/012:30:030
3020299140,0cyclictest0-21swapper/009:08:480
3020299140,0cyclictest0-21swapper/007:15:450
30204991313,0cyclictest0-21swapper/209:25:082
30203991313,0cyclictest0-21swapper/112:00:121
30203991313,0cyclictest0-21swapper/110:52:221
30202991313,0cyclictest0-21swapper/010:42:480
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional