You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 14:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Mon Nov 24, 2025 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
242293399501,47cyclictest2537625-21unin-run21:00:010
2422938994743,3cyclictest2495658-21kworker/u8:1+events_unbound20:20:021
2422933994741,4cyclictest2527253-21kworker/u8:2+flush-259:020:55:000
2422938994340,2cyclictest2635807-21kworker/u8:1+events_unbound22:35:251
2422938994239,2cyclictest2475088-21kworker/u8:0+events_unbound21:10:151
2422938994238,3cyclictest2573614-21kworker/u8:4+events_unbound22:10:211
2422938994238,3cyclictest2527253-21kworker/u8:2+events_unbound20:55:141
2422944994135,4cyclictest2719015-21latency_hist23:54:593
2422938994138,2cyclictest2651244-21kworker/u8:0+events_unbound22:50:241
2422940993935,3cyclictest2552716-21kworker/u8:3+flush-259:021:54:592
2422938993932,5cyclictest2422870-21latency_hist19:10:011
2422940993834,2cyclictest2459216-21kworker/u8:4+flush-259:020:40:172
2422940993830,7cyclictest2433477-21kworker/u8:2+flush-259:020:05:002
2422933993834,3cyclictest2573612-21kworker/u8:2+flush-259:021:49:590
2422938993633,2cyclictest2552716-21kworker/u8:3+events_unbound21:20:181
2422933993633,2cyclictest2734887-21kworker/u8:3+flush-259:000:25:190
2422940993531,3cyclictest2573614-21kworker/u8:4+flush-259:022:10:182
2422940993529,4cyclictest2485331-21base6420:10:012
2422940993528,5cyclictest2495658-21kworker/u8:1+flush-259:020:45:172
2422938993532,2cyclictest2661740-21kworker/u8:4+events_unbound23:00:121
2422938993532,2cyclictest2563312-21kworker/u8:0+events_unbound21:25:211
2422938993532,2cyclictest2475088-21kworker/u8:0+events_unbound21:00:261
2422938993532,2cyclictest2433477-21kworker/u8:2+events_unbound20:05:221
2422938993431,2cyclictest2635807-21kworker/u8:1+events_unbound23:20:171
2422938993430,3cyclictest2672063-21kworker/u8:3+events_unbound23:15:141
2422938993427,5cyclictest2469476-21cp19:55:011
2422938993427,5cyclictest2469476-21cp19:55:001
2422938993330,2cyclictest2495658-21kworker/u8:1+events_unbound21:34:161
2422938993330,2cyclictest2459216-21kworker/u8:4+events_unbound20:40:231
2422944993225,5cyclictest2698871-21systemd-journal21:40:133
2422938993229,2cyclictest2433477-21kworker/u8:2+events_unbound19:31:221
2422933993228,3cyclictest2475088-21kworker/u8:0+flush-259:021:00:190
2422933993227,3cyclictest2563312-21kworker/u8:0+flush-259:021:30:150
2422940993127,3cyclictest2459216-21kworker/u8:4+flush-259:019:50:192
2422940993127,3cyclictest2459216-21kworker/u8:4+flush-259:019:50:192
2422938993128,2cyclictest2573612-21kworker/u8:2+events_unbound22:40:181
2422938993125,4cyclictest3000063-21rs:main1
2422933993127,3cyclictest2464399-21apt-get19:50:130
2422933993127,3cyclictest2464399-21apt-get19:50:120
2422940993024,4cyclictest2605169-21/usr/sbin/munin22:05:122
2422938993027,2cyclictest2734887-21kworker/u8:3+events_unbound00:10:041
2422938993027,2cyclictest2635807-21kworker/u8:1+events_unbound23:25:221
2422938993027,2cyclictest2573612-21kworker/u8:2+events_unbound22:55:271
2422938993027,2cyclictest2573612-21kworker/u8:2+events_unbound22:05:221
2422938993027,2cyclictest2475088-21kworker/u8:0+events_unbound20:30:251
2422938993027,2cyclictest2407726-21kworker/u8:3+events_unbound19:55:041
2422938993027,2cyclictest2402411-21kworker/u8:4+events_unbound19:10:221
2422938993026,3cyclictest2552716-21kworker/u8:3+events_unbound22:00:211
2422933993026,3cyclictest2635807-21kworker/u8:1+flush-259:023:35:160
2422933993025,3cyclictest2442840-21kworker/u8:0+flush-259:019:30:190
2422940992926,2cyclictest2635807-21kworker/u8:1+flush-259:000:15:262
2422940992923,4cyclictest2458151-21ntpq19:40:232
2422938992925,3cyclictest2552716-21kworker/u8:3+events_unbound21:56:221
2422938992826,1cyclictest2495658-21kworker/u8:1+events_unbound21:18:141
2422938992825,2cyclictest2734887-21kworker/u8:3+events_unbound00:35:161
2422938992825,2cyclictest2651244-21kworker/u8:0+events_unbound23:45:261
2422938992825,2cyclictest2635807-21kworker/u8:1+events_unbound22:45:251
2422938992825,2cyclictest2573612-21kworker/u8:2+events_unbound21:41:151
2422938992824,3cyclictest2651244-21kworker/u8:0+events_unbound23:50:181
2422944992722,3cyclictest2578753-21apt-get21:40:003
2422944992720,5cyclictest2656352-21latency_hist22:55:013
2422940992723,3cyclictest2553259-21apt-get21:15:122
2422938992724,2cyclictest2734887-21kworker/u8:3+events_unbound00:25:251
2422938992723,3cyclictest2573612-21kworker/u8:2+events_unbound21:35:231
2422933992724,2cyclictest2563312-21kworker/u8:0+flush-259:021:25:200
2422933992723,3cyclictest2475088-21kworker/u8:0+flush-259:020:10:190
2422933992721,4cyclictest2501561-21base6420:25:000
2422944992620,4cyclictest2745581-21latency_hist00:20:013
2422944992620,4cyclictest2698871-21systemd-journal20:25:503
2422938992623,2cyclictest2573612-21kworker/u8:2+events_unbound23:06:251
2422938992623,2cyclictest2459216-21kworker/u8:4+events_unbound20:15:251
2422938992623,2cyclictest2438674-21apt-get19:25:131
2422938992618,6cyclictest2698871-21systemd-journal23:15:011
242293899261,23cyclictest1-21systemd21:49:591
2422933992622,3cyclictest2635807-21kworker/u8:1+events_unbound22:45:230
2422944992519,4cyclictest2760550-21ntpq00:30:233
2422940992522,2cyclictest503-21systemd-logind20:00:002
2422940992518,6cyclictest2604941-21munin-run22:05:002
2422940992516,6cyclictest2453851-21latency_hist19:40:002
2422938992522,2cyclictest2635807-21kworker/u8:1+events_unbound23:39:561
2422938992522,2cyclictest2635807-21kworker/u8:1+events_unbound00:34:551
2422938992522,2cyclictest2635807-21kworker/u8:1+events_unbound00:20:141
2422938992522,2cyclictest2573612-21kworker/u8:2+events_unbound22:23:231
2422938992521,3cyclictest2495658-21kworker/u8:1+events_unbound21:07:141
2422933992520,3cyclictest2433477-21kworker/u8:2+flush-259:019:25:200
2422933992519,4cyclictest2698871-21systemd-journal00:40:000
2422933992518,5cyclictest2615183-21latency_hist22:15:000
2422944992418,4cyclictest2635807-21kworker/u8:1+flush-259:022:35:173
2422940992419,3cyclictest2459226-21/usr/sbin/munin19:45:162
2422938992421,2cyclictest2734887-21kworker/u8:3+events_unbound00:16:421
2422938992421,2cyclictest2651244-21kworker/u8:0+events_unbound23:31:251
242294499239,2cyclictest2589498-21idleruntime-cro21:50:003
242294499238,3cyclictest2708725-21sh23:45:003
2422944992316,5cyclictest2558152-21latency_hist21:20:013
2422940992318,3cyclictest2552716-21kworker/u8:3+flush-259:021:25:002
2422940992317,4cyclictest2687583-21/usr/sbin/munin23:25:212
2422938992320,2cyclictest2407726-21kworker/u8:3+events_unbound19:46:151
2422938992319,3cyclictest2459216-21kworker/u8:4+flush-259:020:00:051
2422933992316,5cyclictest2661513-21latency_hist23:00:010
2422944992218,2cyclictest2433477-21kworker/u8:2+flush-259:019:55:183
2422944992217,3cyclictest3000063-21rs:main3
2422944992217,3cyclictest2573614-21kworker/u8:4+flush-259:022:00:193
2422944992213,8cyclictest0-21swapper/300:10:003
2422940992215,5cyclictest2552999-21latency_hist21:15:012
2422940992214,6cyclictest2537599-21latency_hist21:00:002
2422938992219,2cyclictest2475088-21kworker/u8:0+events_unbound20:12:141
2422938992219,2cyclictest2407726-21kworker/u8:3+events_unbound19:19:161
2422938992218,3cyclictest503-21systemd-logind19:25:001
2422933992219,2cyclictest2475088-21kworker/u8:0+flush-259:021:10:000
2422933992218,3cyclictest2573614-21kworker/u8:4+events_unbound22:30:140
2422933992218,3cyclictest2573612-21kworker/u8:2+events_unbound22:40:160
2422933992216,4cyclictest2552716-21kworker/u8:3+events_unbound22:04:590
242294499219,11cyclictest2489878-21ntp_kernel_pll_20:10:243
2422944992116,3cyclictest2735715-21diskstats00:10:193
2422940992117,3cyclictest2698871-21systemd-journal23:09:562
2422940992117,3cyclictest2656583-21kworker/u8:3+flush-259:022:55:172
2422940992115,4cyclictest2584761-21diskstats21:45:142
2422933992118,2cyclictest2635807-21kworker/u8:1+flush-259:023:20:200
2422933992118,2cyclictest2407726-21kworker/u8:3+flush-259:019:45:190
2422933992117,3cyclictest2651244-21kworker/u8:0+flush-259:023:30:250
2422933992117,3cyclictest2459216-21kworker/u8:4+flush-259:020:40:250
2422933992117,3cyclictest2433477-21kworker/u8:2+flush-259:020:05:190
242294499208,1cyclictest0-21swapper/321:00:003
2422944992018,1cyclictest0-21swapper/320:25:003
2422944992015,3cyclictest2672063-21kworker/u8:3+flush-259:023:15:163
2422944992014,4cyclictest2755846-21awk00:30:003
2422944992010,8cyclictest2461541-21kthreadcore19:45:203
2422940992016,3cyclictest2579011-21latency_hist21:40:002
2422940992016,3cyclictest2475088-21kworker/u8:0+flush-259:020:10:182
2422940992013,5cyclictest2438408-21latency_hist19:25:002
242294099200,3cyclictest3002-21cron00:25:002
2422938992017,2cyclictest2651244-21kworker/u8:0+events_unbound00:00:001
2422938992017,2cyclictest2635807-21kworker/u8:1+events_unbound23:56:251
2422938992016,3cyclictest2620329-21kworker/u8:1+events_unbound22:25:021
2422933992017,2cyclictest2573612-21kworker/u8:2+flush-259:022:25:270
2422933992017,2cyclictest2573612-21kworker/u8:2+flush-259:021:55:260
2422933992016,3cyclictest2635807-21kworker/u8:1+events_unbound22:35:230
2422933992016,2cyclictest2495658-21kworker/u8:1+events_unbound21:10:220
2422933992013,5cyclictest2594612-21latency_hist21:55:010
2422933992012,5cyclictest2698871-21systemd-journal21:20:130
2422940991913,4cyclictest2428297-21/usr/sbin/munin19:15:132
2422938991916,2cyclictest2573614-21kworker/u8:4+events_unbound22:34:231
2422933991916,2cyclictest2495658-21kworker/u8:1+events_unbound20:35:150
2422933991916,2cyclictest2459216-21kworker/u8:4+flush-259:020:15:220
2422933991915,3cyclictest2734887-21kworker/u8:3+events_unbound00:10:240
2422933991915,3cyclictest2672063-21kworker/u8:3+events_unbound23:10:190
2422933991915,3cyclictest2635807-21kworker/u8:1+events_unbound00:20:120
2422933991915,3cyclictest2433477-21kworker/u8:2+events_unbound19:20:190
2422933991914,4cyclictest2428323-21apt-get19:15:110
2422944991815,2cyclictest0-21swapper/323:00:013
2422944991814,3cyclictest2703507-21latency_hist23:40:013
2422940991812,4cyclictest2703201-21apt-get23:40:002
2422938991815,2cyclictest2527253-21kworker/u8:2+events_unbound20:50:421
2422938991815,2cyclictest2495658-21kworker/u8:1+events_unbound20:37:571
2422938991815,2cyclictest2475088-21kworker/u8:0+events_unbound20:46:551
2422938991813,4cyclictest2707811-21ntpq23:40:211
2422933991815,2cyclictest2433477-21kworker/u8:2+events_unbound20:00:220
2422933991814,3cyclictest2672063-21kworker/u8:3+flush-259:023:15:210
2422933991814,3cyclictest2635807-21kworker/u8:1+events_unbound23:55:250
2422933991812,4cyclictest2506845-21/usr/sbin/munin20:30:140
242294499177,9cyclictest2635807-21kworker/u8:1+flush-259:023:05:003
2422944991715,1cyclictest2522098-21sh20:45:003
2422944991715,1cyclictest0-21swapper/321:05:003
2422944991714,2cyclictest2527229-21sh20:50:003
2422944991710,5cyclictest2573612-21kworker/u8:2+flush-259:021:50:163
2422940991714,2cyclictest2635807-21kworker/u8:1+flush-259:023:50:202
2422940991712,3cyclictest2579315-21gpgv21:40:012
2422940991710,5cyclictest2698871-21systemd-journal20:30:002
2422938991714,2cyclictest2635807-21kworker/u8:1+events_unbound00:07:121
2422938991714,2cyclictest2573612-21kworker/u8:2+events_unbound22:19:321
2422938991714,2cyclictest2495658-21kworker/u8:1+events_unbound20:29:561
2422938991714,2cyclictest2407726-21kworker/u8:3+events_unbound19:44:161
2422933991714,2cyclictest2563312-21kworker/u8:0+flush-259:021:35:190
2422933991713,3cyclictest2698871-21systemd-journal19:59:550
2422933991713,2cyclictest2635807-21kworker/u8:1+flush-259:000:35:000
2422933991712,3cyclictest2698871-21systemd-journal19:13:140
242294499168,6cyclictest2698871-21systemd-journal22:15:003
242294499168,6cyclictest2661740-21kworker/u8:4+flush-259:023:10:013
2422944991613,2cyclictest2609185-21ntp_kernel_pll_22:05:203
2422944991611,3cyclictest2630914-21apt-get22:30:113
2422940991612,3cyclictest2735228-21base6400:10:012
2422940991612,2cyclictest2573612-21kworker/u8:2+flush-259:022:34:592
2422933991614,2cyclictest61450irq/134-enp0s29f1:tx-022:05:160
2422933991614,1cyclictest60650irq/133-enp0s29f1:rx-022:54:310
2422933991613,2cyclictest2661740-21kworker/u8:4+events_unbound23:00:180
2422933991612,3cyclictest2651244-21kworker/u8:0+events_unbound00:00:200
2422933991612,3cyclictest2635807-21kworker/u8:1+events_unbound00:05:270
2422933991612,3cyclictest2552716-21kworker/u8:3+events_unbound21:15:190
2422933991612,3cyclictest2433477-21kworker/u8:2+events_unbound19:35:200
2422933991611,3cyclictest2651244-21kworker/u8:0+flush-259:023:30:010
2422933991610,4cyclictest2501046-21latency_hist20:25:000
242294499158,5cyclictest2619417-21ntpq22:15:223
2422944991511,3cyclictest2651244-21kworker/u8:0+flush-259:023:26:333
2422944991511,3cyclictest2495658-21kworker/u8:1+flush-259:021:05:203
2422940991512,2cyclictest2625485-21systemd-run22:25:002
2422940991510,4cyclictest2719677-21munin-node23:55:122
2422940991510,4cyclictest2677232-21/usr/sbin/munin23:15:142
2422940991510,3cyclictest2698871-21systemd-journal20:51:152
2422940991510,3cyclictest2672063-21kworker/u8:3+flush-259:023:15:002
242293399159,4cyclictest2708733-21latency_hist23:45:000
2422933991513,1cyclictest60650irq/133-enp0s29f1:rx-023:05:110
2422933991513,1cyclictest60650irq/133-enp0s29f1:rx-000:18:200
2422933991512,2cyclictest60650irq/133-enp0s29f1:rx-022:16:190
2422933991511,3cyclictest2651244-21kworker/u8:0+events_unbound23:50:200
242294099148,4cyclictest2698871-21systemd-journal00:05:002
2422940991411,2cyclictest2651244-21kworker/u8:0+flush-259:022:50:172
2422940991411,2cyclictest2459216-21kworker/u8:4+flush-259:020:36:362
2422940991411,2cyclictest0-21swapper/200:40:002
2422940991410,3cyclictest2645111-21needreboot22:40:192
2422940991410,3cyclictest2495658-21kworker/u8:1+flush-259:021:05:262
2422933991413,1cyclictest60650irq/133-enp0s29f1:rx-019:40:200
242294499139,3cyclictest2479999-21latency_hist20:05:003
242294499138,3cyclictest2625734-21apt-get22:25:153
2422944991310,2cyclictest2567973-21unixbench_multi21:25:263
2422944991310,2cyclictest0-21swapper/319:10:143
242294099139,3cyclictest2573612-21kworker/u8:2+flush-259:022:26:482
242294099138,3cyclictest2703712-21base6423:40:012
2422940991310,2cyclictest2731032-21kthreadcore00:05:212
2422940991310,1cyclictest0-21swapper/223:20:202
2422940991310,1cyclictest0-21swapper/220:15:212
242294099130,2cyclictest2666681-21awk23:05:012
242293399139,3cyclictest2573612-21kworker/u8:2+flush-259:022:20:190
2422933991311,1cyclictest60650irq/133-enp0s29f1:rx-023:45:230
2422933991311,1cyclictest60650irq/133-enp0s29f1:rx-020:45:170
242294499129,2cyclictest2443879-21cpuspeed_turbos19:30:143
242294499129,2cyclictest0-21swapper/321:30:193
242294499128,3cyclictest2646466-21cut22:45:113
242294499128,3cyclictest2635807-21kworker/u8:1+flush-259:023:58:553
242294499128,3cyclictest2552716-21kworker/u8:3+flush-259:021:20:263
242294499128,3cyclictest2475088-21kworker/u8:0+flush-259:020:31:343
242294499126,4cyclictest492-21dbus-daemon23:35:013
242294099129,2cyclictest2693171-21irqcore23:30:172
242294099129,2cyclictest2542334-21unixbench_singl21:00:282
242294099128,3cyclictest2734887-21kworker/u8:3+flush-259:000:30:242
242294099128,3cyclictest0-21swapper/222:15:242
242294099128,2cyclictest0-21swapper/219:30:272
242294099126,4cyclictest2568424-21latency_hist21:30:002
242294499118,2cyclictest2449147-21fschecks_count19:35:143
242294499118,2cyclictest0-21swapper/320:10:003
242294499117,3cyclictest2433512-21apt-get19:20:133
242294099119,1cyclictest0-21swapper/221:30:132
242294099119,1cyclictest0-21swapper/220:25:012
242294499108,1cyclictest0-21swapper/319:50:153
242294499108,1cyclictest0-21swapper/319:50:153
242294499107,2cyclictest2621590-21tr22:20:173
242294499107,2cyclictest0-21swapper/320:20:003
242294499106,3cyclictest503-21systemd-logind20:40:013
242294499106,3cyclictest2443227-21sshd19:26:143
242294499104,5cyclictest2713502-21users23:45:243
242294099105,4cyclictest2647056-21sed22:45:152
242294099104,4cyclictest2594850-21/usr/sbin/munin21:55:212
24229449996,2cyclictest2766086-21unixbench_multi00:35:253
24229449994,4cyclictest492-21dbus-daemon23:15:003
24229449994,4cyclictest2428483-21diskstats19:15:133
24229449985,2cyclictest2594975-21cpuspeed_turbos21:55:143
24229449985,1cyclictest0-21swapper/321:10:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional