You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 09:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sat Apr 11, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3523665994741,4cyclictest3705105-21latency_hist22:05:001
3523657994639,5cyclictest3831000-21cat00:05:020
3523665994439,4cyclictest3607269-21diskstats20:30:141
3523671994236,4cyclictest3842006-21latency_hist00:15:003
3523667994138,2cyclictest3663483-21kworker/u8:3+flush-259:021:30:182
3523667994035,3cyclictest252-21systemd-journal21:19:392
3523671993933,5cyclictest3590461-21ntpq20:10:213
3523657993933,4cyclictest3626617-21ntpq20:45:210
3523665993834,3cyclictest3586144-21kworker/u8:1+flush-259:020:25:201
3523657993832,4cyclictest3549879-21cp19:35:010
3523665993529,4cyclictest3555081-21latency_hist19:40:011
3523657993530,3cyclictest3695064-21apt-get21:55:110
3523671993430,3cyclictest3490275-21kworker/u8:0+flush-259:019:25:003
3523665993430,3cyclictest3612053-21kworker/u8:2+flush-259:020:45:171
3523667993228,3cyclictest3804538-21kworker/u8:0+flush-259:000:30:152
3523657993226,4cyclictest3668625-21apt-get21:30:010
3523667993127,3cyclictest3622567-21/usr/sbin/munin20:45:212
3523667993127,3cyclictest3586145-21kworker/u8:4+flush-259:020:10:172
3523667993126,3cyclictest3550136-21base6419:35:022
3523665993125,4cyclictest3751765-21kworker/u8:4+flush-259:023:45:001
3523667993024,4cyclictest3830839-21latency_hist00:05:012
3523657993024,4cyclictest3751765-21kworker/u8:4+flush-259:023:30:170
3523671992923,4cyclictest3638145-21diskstats21:00:143
3523665992925,3cyclictest3674045-21kworker/u8:0+flush-259:021:35:171
3523665992923,4cyclictest3710206-21awk22:10:001
3523657992925,3cyclictest3842248-21/usr/sbin/munin00:15:130
3523657992823,3cyclictest3771481-21kworker/u8:0+flush-259:023:10:160
3523671992722,3cyclictest252-21systemd-journal20:15:383
3523667992722,4cyclictest3809430-21latency_hist23:45:012
3523665992722,3cyclictest3689544-21kworker/u8:1+flush-259:022:15:171
3523665992721,4cyclictest252-21systemd-journal19:25:391
3523665992720,5cyclictest3852328-21latency_hist00:25:001
3523657992721,4cyclictest3585250-21ntpq20:05:210
3523667992622,3cyclictest3689544-21kworker/u8:1+flush-259:022:35:112
3523665992618,5cyclictest252-21systemd-journal20:20:001
3523671992521,3cyclictest3788474-21base6423:25:023
3523667992519,4cyclictest3679772-21diskstats21:40:132
3523665992518,5cyclictest3825966-21apt-get00:00:131
3523657992519,4cyclictest3601965-21apt-get20:25:110
3523671992421,2cyclictest3469540-21kworker/u8:2+flush-259:019:30:233
3523667992420,2cyclictest3674045-21kworker/u8:0+flush-259:021:35:162
3523665992418,4cyclictest3857700-21/usr/sbin/munin00:30:131
3523665992418,4cyclictest3857465-21latency_hist00:30:001
3523665992418,4cyclictest3736570-21base6422:35:011
3523657992419,3cyclictest3836596-21base6400:10:010
3523657992418,4cyclictest252-21systemd-journal23:09:400
352365799241,21cyclictest1-21systemd23:45:400
3523671992319,3cyclictest3657940-21kworker/u8:2+flush-259:021:25:263
3523665992319,3cyclictest3757241-21kworker/u8:1+flush-259:023:50:001
3523665992319,3cyclictest3751765-21kworker/u8:4+flush-259:023:50:151
3523665992317,4cyclictest3469540-21kworker/u8:2+flush-259:019:40:271
3523657992317,4cyclictest3862614-21latency_hist00:35:000
3523671992216,5cyclictest3803848-21apt-get23:40:003
3523667992217,3cyclictest252-21systemd-journal00:08:122
3523665992218,3cyclictest3586145-21kworker/u8:4+flush-259:020:10:241
3523665992218,3cyclictest3469540-21kworker/u8:2+flush-259:019:30:201
3523665992217,3cyclictest3771481-21kworker/u8:0+events_unbound23:10:201
3523665992217,3cyclictest3637761-21kworker/u8:0+events_unbound21:10:171
3523671992117,3cyclictest3731037-21/usr/sbin/munin22:30:143
3523667992117,3cyclictest3575389-21kworker/u8:2+flush-259:020:20:162
3523667992115,4cyclictest3612266-21/usr/sbin/munin20:35:212
3523665992116,4cyclictest3663483-21kworker/u8:3+events_unbound21:40:181
3523665992115,4cyclictest3657940-21kworker/u8:2+events_unbound21:50:141
3523671992016,3cyclictest3689544-21kworker/u8:1+flush-259:022:15:213
3523671992014,4cyclictest252-21systemd-journal19:45:003
3523667992016,3cyclictest3657940-21kworker/u8:2+flush-259:021:20:172
3523667992016,3cyclictest3617778-21kworker/u8:1+flush-259:021:00:252
3523667992016,3cyclictest3612053-21kworker/u8:2+flush-259:021:05:222
3523665992016,3cyclictest3554579-21kworker/u8:3+flush-259:019:49:591
3523665992016,2cyclictest3715337-21cron22:15:001
3523665992014,4cyclictest3735961-21latency_hist22:35:011
3523657992015,3cyclictest3787962-21apt-get23:25:000
3523657992014,4cyclictest252-21systemd-journal22:16:400
3523657992014,4cyclictest252-21systemd-journal21:00:390
3523671991916,2cyclictest3671589-21kthreadcore21:30:203
3523671991916,2cyclictest3580959-21sh20:05:003
3523671991915,3cyclictest3819718-21kworker/u8:2+flush-259:000:20:163
3523671991915,3cyclictest3689544-21kworker/u8:1+flush-259:022:25:013
3523671991915,3cyclictest3684716-21/usr/sbin/munin21:45:143
3523671991915,3cyclictest3657940-21kworker/u8:2+flush-259:022:00:173
3523667991916,2cyclictest3741811-21apt-get22:40:112
3523665991916,2cyclictest3490275-21kworker/u8:0+flush-259:019:10:201
3523665991915,3cyclictest3751765-21kworker/u8:4+events_unbound22:50:231
3523665991914,3cyclictest3490275-21kworker/u8:0+flush-259:019:20:211
3523665991913,4cyclictest3663483-21kworker/u8:3+flush-259:021:30:201
3523671991815,2cyclictest509-21rs:main3
3523671991814,3cyclictest3751765-21kworker/u8:4+flush-259:000:05:003
3523667991814,3cyclictest3657940-21kworker/u8:2+flush-259:021:46:122
3523667991814,2cyclictest0-21swapper/219:45:002
3523665991814,3cyclictest3804538-21kworker/u8:0+flush-259:000:35:201
3523665991814,3cyclictest3657940-21kworker/u8:2+events_unbound21:55:231
3523665991814,3cyclictest3602284-21kworker/u8:0+flush-259:020:35:201
3523665991814,3cyclictest3586145-21kworker/u8:4+events_unbound20:40:201
3523665991812,4cyclictest3657940-21kworker/u8:2+flush-259:022:20:101
3523665991811,5cyclictest3836445-21latency_hist00:10:011
3523657991815,2cyclictest3751765-21kworker/u8:4+events_unbound23:40:000
3523657991812,4cyclictest3736810-21diskstats22:35:130
3523657991812,4cyclictest252-21systemd-journal23:00:000
3523671991713,3cyclictest3788374-21gpgv23:25:013
3523671991711,4cyclictest3752273-21diskstats22:50:123
3523667991713,3cyclictest509-21rs:main2
3523667991711,4cyclictest509-21rs:main2
3523667991711,4cyclictest252-21systemd-journal22:12:402
3523665991714,2cyclictest3571883-21kthreadcore19:55:161
3523665991712,3cyclictest252-21systemd-journal00:15:011
3523657991713,3cyclictest3819718-21kworker/u8:2+flush-259:023:55:200
3523671991613,2cyclictest3657940-21kworker/u8:2+flush-259:022:10:003
3523671991613,2cyclictest3608129-21kthreadcore20:30:163
3523671991611,4cyclictest3867813-21latency_hist00:40:003
3523671991611,3cyclictest509-21rs:main3
3523667991614,1cyclictest0-21swapper/223:40:012
3523667991612,3cyclictest3804538-21kworker/u8:0+flush-259:000:10:162
3523667991611,3cyclictest3689544-21kworker/u8:1+flush-259:022:35:002
352366599169,5cyclictest3612053-21kworker/u8:2+flush-259:021:02:481
3523665991612,3cyclictest3685757-21kthreadcore21:45:171
3523665991612,3cyclictest3657940-21kworker/u8:2+events_unbound22:25:151
3523665991612,2cyclictest0-21swapper/123:15:181
3523665991611,4cyclictest3804538-21kworker/u8:0+flush-259:000:10:211
3523665991611,3cyclictest3575993-21/usr/sbin/munin20:00:151
3523665991610,4cyclictest3803851-21latency_hist23:40:021
3523657991613,2cyclictest60350irq/132-enp0s29f1:rx-022:20:150
3523657991613,2cyclictest60350irq/132-enp0s29f1:rx-021:05:200
3523657991613,2cyclictest3469540-21kworker/u8:2+flush-259:019:30:000
3523657991612,3cyclictest509-21rs:main0
3523671991512,2cyclictest3857448-21cron00:29:593
3523671991512,2cyclictest3694791-21idleruntime-cro21:55:003
3523671991512,2cyclictest2971-21cron21:15:003
3523671991511,3cyclictest509-21rs:main3
3523671991511,3cyclictest3842011-21turbostat00:19:543
3523671991511,3cyclictest3751765-21kworker/u8:4+flush-259:000:08:173
3523671991510,3cyclictest3804508-21/usr/sbin/munin23:40:193
3523667991512,2cyclictest3726323-21irqcore22:25:172
3523667991512,2cyclictest0-21swapper/219:20:002
3523665991511,3cyclictest3560105-21kworker/u8:1+flush-259:019:50:371
3523665991511,2cyclictest0-21swapper/120:10:001
3523665991510,4cyclictest3762795-21kworker/u8:2+events_unbound23:05:151
3523665991510,4cyclictest3648003-21kworker/u8:3+events_unbound21:15:181
352365799159,5cyclictest3554189-21ntpq19:35:220
3523657991513,1cyclictest60350irq/132-enp0s29f1:rx-023:25:120
3523657991513,1cyclictest60350irq/132-enp0s29f1:rx-022:30:230
3523657991513,1cyclictest60350irq/132-enp0s29f1:rx-021:50:160
3523657991513,1cyclictest60350irq/132-enp0s29f1:rx-021:20:150
3523657991513,1cyclictest60350irq/132-enp0s29f1:rx-020:52:170
3523657991513,1cyclictest60350irq/132-enp0s29f1:rx-020:40:020
3523657991513,1cyclictest60350irq/132-enp0s29f1:rx-020:20:440
3523657991513,1cyclictest60350irq/132-enp0s29f1:rx-019:50:160
3523657991513,1cyclictest60350irq/132-enp0s29f1:rx-019:12:260
3523657991512,2cyclictest60350irq/132-enp0s29f1:rx-023:15:300
3523657991512,2cyclictest60350irq/132-enp0s29f1:rx-023:03:490
3523657991512,2cyclictest60350irq/132-enp0s29f1:rx-022:45:140
3523657991512,2cyclictest60350irq/132-enp0s29f1:rx-022:45:140
3523657991512,2cyclictest60350irq/132-enp0s29f1:rx-021:47:410
3523657991512,2cyclictest60350irq/132-enp0s29f1:rx-021:40:420
3523657991512,2cyclictest60350irq/132-enp0s29f1:rx-021:37:540
3523657991512,2cyclictest60350irq/132-enp0s29f1:rx-020:10:110
3523657991512,2cyclictest60350irq/132-enp0s29f1:rx-000:22:260
3523657991512,2cyclictest60350irq/132-enp0s29f1:rx-000:10:030
3523657991511,3cyclictest3847199-21kworker/u8:1+flush-259:000:25:250
3523657991511,3cyclictest3751765-21kworker/u8:4+flush-259:023:50:230
3523657991511,2cyclictest60350irq/132-enp0s29f1:rx-000:35:200
352367199148,5cyclictest3818826-21ntpq23:50:203
3523671991411,2cyclictest3695476-21tr21:55:163
3523671991411,2cyclictest3603607-21kthreadcore20:25:173
3523671991411,2cyclictest3528790-21idleruntime-cro19:15:013
352366799148,4cyclictest252-21systemd-journal00:25:002
352366599147,4cyclictest3523602-21latency_hist19:10:011
352366599146,7cyclictest3783325-21uniq23:20:141
352366599142,4cyclictest3746714-21munin-run22:45:001
3523665991410,3cyclictest3759557-21kthreadcore22:55:161
3523665991410,3cyclictest3586144-21kworker/u8:1+flush-259:020:25:001
3523657991413,1cyclictest60350irq/132-enp0s29f1:rx-022:29:300
3523657991413,1cyclictest60350irq/132-enp0s29f1:rx-019:57:490
3523657991412,2cyclictest60350irq/132-enp0s29f1:rx-019:20:150
3523657991412,1cyclictest60350irq/132-enp0s29f1:rx-022:00:140
3523657991412,1cyclictest60350irq/132-enp0s29f1:rx-021:15:140
3523657991412,1cyclictest60350irq/132-enp0s29f1:rx-020:01:260
3523657991412,1cyclictest60350irq/132-enp0s29f1:rx-019:45:250
3523657991412,1cyclictest60350irq/132-enp0s29f1:rx-019:42:570
3523657991411,2cyclictest60350irq/132-enp0s29f1:rx-022:40:180
3523657991411,2cyclictest60350irq/132-enp0s29f1:rx-020:15:370
3523657991411,2cyclictest60350irq/132-enp0s29f1:rx-019:15:180
352367199139,3cyclictest3725639-21turbostat22:29:553
3523671991310,2cyclictest3751765-21kworker/u8:4+flush-259:023:45:453
3523671991310,2cyclictest3658637-21cpuspeed_turbos21:20:143
3523671991310,2cyclictest3612053-21kworker/u8:2+flush-259:020:50:153
3523671991310,1cyclictest0-21swapper/300:00:003
352366799139,3cyclictest3751765-21kworker/u8:4+flush-259:023:51:462
352366799138,4cyclictest252-21systemd-journal22:18:402
352366799138,3cyclictest252-21systemd-journal23:01:402
3523667991311,1cyclictest3847808-21mii-tool00:20:142
3523667991310,2cyclictest3751858-21idleruntime-cro22:50:012
3523667991310,2cyclictest3751858-21idleruntime-cro22:50:002
352366599139,3cyclictest3612053-21kworker/u8:2+events_unbound20:55:221
352366599137,4cyclictest3757241-21kworker/u8:1+flush-259:023:35:001
352366599131,10cyclictest3751847-21awk22:50:001
352366599131,10cyclictest3751847-21awk22:50:001
3523665991310,2cyclictest3658980-21irqcore21:20:171
3523665991310,2cyclictest3628153-21irqcore20:50:151
352365799139,3cyclictest3804533-21apt-get23:40:100
3523657991312,1cyclictest60350irq/132-enp0s29f1:rx-022:13:160
3523657991311,2cyclictest60350irq/132-enp0s29f1:rx-020:39:140
3523657991311,1cyclictest60350irq/132-enp0s29f1:rx-021:30:260
3523657991310,2cyclictest60350irq/132-enp0s29f1:rx-022:54:520
352367199129,2cyclictest3601267-21unixbench_singl20:20:253
352367199127,4cyclictest3602284-21kworker/u8:0+flush-259:020:35:103
352367199126,5cyclictest252-21systemd-journal20:45:003
352367199126,4cyclictest3747490-21sed22:45:173
352367199126,4cyclictest3747490-21sed22:45:173
352366799129,2cyclictest0-21swapper/220:50:182
352366799128,3cyclictest3757241-21kworker/u8:1+flush-259:023:30:252
352366799128,3cyclictest3751765-21kworker/u8:4+flush-259:022:50:232
352366799128,3cyclictest1-21systemd00:20:002
352366799126,4cyclictest252-21systemd-journal23:24:402
352366799121,2cyclictest3575768-21basename20:00:002
3523667991210,1cyclictest0-21swapper/222:25:002
3523667991210,1cyclictest0-21swapper/219:35:002
352366599129,2cyclictest3643344-21fschecks_count21:05:141
352366599128,3cyclictest3657940-21kworker/u8:2+flush-259:021:25:161
3523657991210,1cyclictest60350irq/132-enp0s29f1:rx-020:30:150
352367199119,1cyclictest3775023-21tr23:10:183
352367199119,1cyclictest0-21swapper/323:15:213
352367199117,3cyclictest3655716-21tr21:15:173
352367199111,2cyclictest3772444-21latency_hist23:10:003
352367199110,9cyclictest3746854-21cat22:45:003
352366799118,2cyclictest3777203-21unixbench_singl23:10:242
352366799116,4cyclictest3777869-21/usr/sbin/munin23:15:182
352366799116,4cyclictest3469540-21kworker/u8:2+flush-259:019:45:262
352366599117,2cyclictest3757241-21kworker/u8:1+events_unbound23:00:161
352367199107,2cyclictest3617778-21kworker/u8:1+flush-259:020:46:113
352366799107,2cyclictest3632937-21cpuspeed_turbos20:55:112
352366799107,2cyclictest3570120-21unixbench_multi19:50:252
352366799106,3cyclictest3657940-21kworker/u8:2+flush-259:021:25:102
352366599108,1cyclictest3529478-21sed19:15:161
352366599105,3cyclictest3821709-21cut23:55:161
35236679996,1cyclictest0-21swapper/220:30:222
35236679986,1cyclictest0-21swapper/220:05:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional