You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-13 - 12:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat Dec 13, 2025 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3336630994139,1cyclictest3346994-21kworker/0:1+pm20:22:010
3336630993935,3cyclictest3544741-21kworker/u8:5+events_unbound22:50:180
3336636993630,4cyclictest3414533-21/usr/sbin/munin20:25:133
3336636993530,4cyclictest3648623-21kworker/u8:2+flush-259:000:30:133
3336630993532,2cyclictest3482731-21kworker/u8:3+events_unbound21:40:220
3336630993431,2cyclictest3414035-21kworker/u8:2+events_unbound20:30:150
3336631993228,2cyclictest3440395-21kworker/u8:2+flush-259:021:40:171
3336630993223,6cyclictest252-21systemd-journal00:15:000
3336630993223,6cyclictest252-21systemd-journal00:14:590
3336630993128,2cyclictest3529109-21kworker/u8:0+events_unbound23:45:200
3336630993128,2cyclictest3357268-21kworker/u8:1+events_unbound19:55:230
3336630993027,2cyclictest3513698-21kworker/u8:2+events_unbound23:05:130
3336630992926,2cyclictest3648623-21kworker/u8:2+events_unbound00:15:160
3336630992926,2cyclictest3648623-21kworker/u8:2+events_unbound00:15:160
3336630992825,2cyclictest3607115-21kworker/u8:2+events_unbound23:30:130
3336630992825,2cyclictest3513698-21kworker/u8:2+events_unbound22:35:230
3336630992825,2cyclictest3456473-21kworker/u8:0+flush-259:021:05:020
3336630992824,3cyclictest3591270-21kworker/u8:1+events_unbound23:20:170
3336633992723,3cyclictest3570813-21kworker/u8:3+flush-259:023:10:012
3336631992723,3cyclictest3612452-21kworker/u8:4+flush-259:023:50:171
3336630992725,1cyclictest3482731-21kworker/u8:3+events_unbound21:35:000
3336630992723,3cyclictest3597058-21kworker/u8:3+flush-259:023:25:140
3336630992623,2cyclictest3513698-21kworker/u8:2+events_unbound22:33:140
3336630992623,2cyclictest3325586-21kworker/u8:3+events_unbound19:10:130
3336630992623,2cyclictest1252593-21kworker/u8:0+flush-259:019:20:190
3336630992622,3cyclictest3534665-21apt-get22:20:140
3336630992620,4cyclictest252-21systemd-journal20:50:130
3336633992522,2cyclictest3414035-21kworker/u8:2+flush-259:020:40:002
3336631992519,4cyclictest3373026-21latency_hist19:45:001
3336630992521,3cyclictest3508553-21kworker/u8:1+events_unbound22:00:190
3336633992417,5cyclictest3675642-21cat00:40:002
3336631992419,3cyclictest3591270-21kworker/u8:1+events_unbound23:15:141
3336630992421,2cyclictest3570813-21kworker/u8:3+events_unbound22:55:190
3336630992421,2cyclictest3498395-21kworker/u8:4+events_unbound21:45:250
3336636992316,4cyclictest3403967-21latency_hist20:15:003
3336631992319,2cyclictest3513698-21kworker/u8:2+events_unbound22:15:101
3336636992217,3cyclictest3654244-21/usr/sbin/munin00:15:103
3336636992217,3cyclictest3654244-21/usr/sbin/munin00:15:103
3336636992215,5cyclictest252-21systemd-journal00:40:003
3336633992219,2cyclictest3648623-21kworker/u8:2+flush-259:000:10:142
3336633992219,2cyclictest3648623-21kworker/u8:2+flush-259:000:10:142
3336633992217,3cyclictest490-21dbus-daemon19:35:002
3336630992219,2cyclictest3633132-21kworker/u8:1+flush-259:000:07:390
3336630992219,2cyclictest3612452-21kworker/u8:4+events_unbound00:30:170
3336630992219,2cyclictest3461852-21kworker/u8:1+flush-259:021:10:180
3336630992219,2cyclictest3440395-21kworker/u8:2+flush-259:021:38:130
3336630992219,2cyclictest3430455-21kworker/u8:1+events_unbound20:40:140
3336630992218,3cyclictest3482731-21kworker/u8:3+events_unbound21:55:140
3336630992218,3cyclictest3482731-21kworker/u8:3+events_unbound21:55:140
3336630992218,3cyclictest3357268-21kworker/u8:1+flush-259:019:51:070
3336630992215,5cyclictest3361813-21ntpq19:30:220
3336636992116,3cyclictest252-21systemd-journal22:02:133
3336633992119,1cyclictest0-21swapper/222:45:012
3336633992118,2cyclictest500-21systemd-logind21:00:002
3336633992117,3cyclictest3352057-21apt-get19:25:012
3336633992115,4cyclictest3643256-21latency_hist00:05:012
3336633992115,4cyclictest3622576-21latency_hist23:44:592
3336631992117,3cyclictest500-21systemd-logind00:40:011
3336631992117,3cyclictest3529109-21kworker/u8:0+flush-259:000:20:211
3336631992117,3cyclictest3529109-21kworker/u8:0+events_unbound22:50:141
3336631992117,2cyclictest3555083-21cron22:40:011
333663199210,19cyclictest1-21systemd22:10:001
3336630992118,2cyclictest3633132-21kworker/u8:1+events_unbound23:57:340
3336630992118,2cyclictest3513698-21kworker/u8:2+events_unbound23:05:010
3336630992118,2cyclictest3498395-21kworker/u8:4+flush-259:022:10:160
3336630992118,2cyclictest3482731-21kworker/u8:3+events_unbound21:50:130
3336630992118,2cyclictest3414035-21kworker/u8:2+flush-259:020:38:090
3336630992118,2cyclictest3357268-21kworker/u8:1+events_unbound19:40:230
3336630992115,5cyclictest252-21systemd-journal19:30:000
3336636992015,3cyclictest3570818-21/usr/sbin/munin22:55:113
3336631992017,2cyclictest3408812-21kworker/u8:0+flush-259:020:20:131
3336631992016,3cyclictest3612452-21kworker/u8:4+flush-259:023:40:151
3336631992016,3cyclictest3544741-21kworker/u8:5+events_unbound22:40:151
3336631992016,3cyclictest3398876-21kworker/u8:4+flush-259:020:40:181
3336631992014,4cyclictest3367855-21latency_hist19:40:001
3336630992017,2cyclictest3529109-21kworker/u8:0+events_unbound22:45:200
3336630992017,2cyclictest3513698-21kworker/u8:2+events_unbound23:14:150
3336630992017,2cyclictest3440395-21kworker/u8:2+events_unbound21:25:210
3336630992017,2cyclictest3362479-21kworker/u8:3+events_unbound20:05:170
3336630992017,2cyclictest3362479-21kworker/u8:3+events_unbound19:35:190
3336630992017,2cyclictest1252593-21kworker/u8:0+events_unbound19:15:240
3336636991915,3cyclictest3346923-21kworker/u8:3+flush-259:019:25:163
3336633991915,3cyclictest3591270-21kworker/u8:1+events_unbound23:15:182
3336633991914,4cyclictest1252593-21kworker/u8:0+flush-259:019:10:162
3336631991913,4cyclictest3465935-21ntpq21:10:221
333663199190,17cyclictest252-21systemd-journal21:40:001
3336630991916,2cyclictest3597058-21kworker/u8:3+events_unbound23:50:230
3336633991815,2cyclictest3513698-21kworker/u8:2+flush-259:022:30:252
3336633991815,2cyclictest3414982-21irqcore20:25:162
3336633991814,3cyclictest3633323-21diskstats23:55:102
3336633991814,3cyclictest3565396-21idleruntime-cro22:50:002
3336633991813,3cyclictest3570813-21kworker/u8:3+flush-259:022:55:132
3336631991815,2cyclictest3666951-21kthreadcore00:25:141
3336631991815,2cyclictest3482731-21kworker/u8:3+flush-259:021:30:191
3336631991815,2cyclictest3378854-21irqcore19:50:161
3336631991815,2cyclictest3360144-21kthreadcore19:30:191
3336631991814,3cyclictest3513698-21kworker/u8:2+flush-259:022:10:131
3336631991814,2cyclictest3529109-21kworker/u8:0+events_unbound23:00:231
3336631991813,4cyclictest3529109-21kworker/u8:0+flush-259:023:10:101
3336630991815,2cyclictest3664401-21kworker/u8:1+events_unbound00:25:190
3336630991815,2cyclictest3398876-21kworker/u8:4+events_unbound21:17:020
3336630991815,2cyclictest3357268-21kworker/u8:1+events_unbound20:10:160
3336630991811,5cyclictest252-21systemd-journal20:26:080
3336636991714,2cyclictest3638063-21dump-pmu-power23:59:593
3336636991710,3cyclictest3591270-21kworker/u8:1+flush-259:023:20:003
3336633991715,1cyclictest3440395-21kworker/u8:2+flush-259:021:00:212
3336631991713,3cyclictest3633132-21kworker/u8:1+events_unbound00:00:151
3336631991713,3cyclictest3612452-21kworker/u8:4+events_unbound00:15:101
3336631991713,3cyclictest3612452-21kworker/u8:4+events_unbound00:15:101
3336631991713,3cyclictest1252593-21kworker/u8:0+flush-259:019:25:211
3336631991712,4cyclictest3612452-21kworker/u8:4+flush-259:023:35:231
3336630991714,2cyclictest60350irq/132-enp0s29f1:rx-020:55:180
3336630991714,2cyclictest3633132-21kworker/u8:1+events_unbound00:02:340
3336630991714,2cyclictest3529109-21kworker/u8:0+events_unbound00:20:150
3336630991714,2cyclictest3408812-21kworker/u8:0+events_unbound20:46:120
3336636991612,3cyclictest3398876-21kworker/u8:4+events_unbound21:05:003
3336636991611,4cyclictest0-21swapper/319:15:193
3336636991610,4cyclictest3472393-21latency_hist21:20:013
3336633991613,2cyclictest3513698-21kworker/u8:2+flush-259:022:10:192
3336633991613,2cyclictest3398876-21kworker/u8:4+flush-259:021:15:302
3336633991612,3cyclictest3378206-21kworker/u8:0+flush-259:019:55:152
3336633991611,3cyclictest3373268-21/usr/sbin/munin19:45:152
3336633991610,4cyclictest3591252-21latency_hist23:15:002
333663399160,7cyclictest3539621-21munin-plugin-st22:25:002
3336631991614,1cyclictest3424639-21cut20:35:001
3336631991613,2cyclictest3482731-21kworker/u8:3+events_unbound21:45:151
3336631991613,2cyclictest3362479-21kworker/u8:3+flush-259:020:15:231
3336631991612,3cyclictest3633132-21kworker/u8:1+flush-259:000:05:211
3336631991612,3cyclictest3508553-21kworker/u8:1+events_unbound22:00:141
3336631991612,3cyclictest3440395-21kworker/u8:2+flush-259:020:55:181
3336631991611,3cyclictest252-21systemd-journal20:03:071
3336630991614,1cyclictest60350irq/132-enp0s29f1:rx-021:03:370
3336630991613,2cyclictest3529406-21kworker/u8:3+flush-259:022:18:140
3336630991612,3cyclictest3612452-21kworker/u8:4+events_unbound23:40:140
3336636991513,1cyclictest0-21swapper/320:00:003
3336636991513,1cyclictest0-21swapper/300:10:003
3336636991512,1cyclictest0-21swapper/321:34:593
3336636991511,3cyclictest3519099-21/usr/sbin/munin22:05:203
3336633991512,2cyclictest3663515-21ntp_kernel_pll_00:20:202
3336633991512,2cyclictest3357268-21kworker/u8:1+flush-259:020:10:002
3336633991512,2cyclictest252-21systemd-journal20:45:012
3336633991512,1cyclictest0-21swapper/223:35:002
3336633991511,2cyclictest3495587-21kthreadcore21:40:172
3336633991510,3cyclictest3357268-21kworker/u8:1+events_unbound20:00:172
333663199159,4cyclictest252-21systemd-journal19:48:071
3336631991512,2cyclictest3547300-21kthreadcore22:30:191
3336631991512,2cyclictest3341748-21sh19:15:001
3336631991511,3cyclictest507-21in:imuxsock00:15:001
3336631991511,3cyclictest507-21in:imuxsock00:15:001
3336631991511,3cyclictest3440395-21kworker/u8:2+events_unbound21:25:231
3336631991511,2cyclictest3440525-21cron20:50:001
3336630991512,2cyclictest3612452-21kworker/u8:4+events_unbound00:37:550
3336630991510,3cyclictest3539869-21apt-get22:25:090
333663699149,3cyclictest3597069-21/usr/sbin/munin23:20:273
3336636991412,1cyclictest0-21swapper/320:40:003
3336636991411,2cyclictest3617894-21fschecks_time23:40:163
3336636991411,2cyclictest3580969-21sh23:05:003
3336636991411,2cyclictest3358877-21kthreadcore19:30:183
3336636991410,3cyclictest252-21systemd-journal21:40:003
3336633991411,2cyclictest3478215-21irqcore21:25:192
3336633991410,3cyclictest3398876-21kworker/u8:4+flush-259:020:20:012
3336633991410,2cyclictest0-21swapper/220:30:172
3336631991411,2cyclictest3357268-21kworker/u8:1+events_unbound20:00:001
3336631991410,3cyclictest3611751-21unixbench_multi23:30:211
3336631991410,3cyclictest3513698-21kworker/u8:2+flush-259:022:55:201
3336631991410,3cyclictest3466796-21apt-get21:15:101
3336631991410,3cyclictest3408812-21kworker/u8:0+flush-259:020:25:071
3336630991413,1cyclictest60350irq/132-enp0s29f1:rx-023:37:270
3336630991412,1cyclictest60350irq/132-enp0s29f1:rx-019:47:550
3336630991411,2cyclictest3550128-21kworker/u8:3+events_unbound22:43:140
3336630991411,2cyclictest3513698-21kworker/u8:2+flush-259:023:17:330
3336630991410,3cyclictest3440395-21kworker/u8:2+flush-259:021:21:560
333663699139,3cyclictest3653587-21apt-get00:15:013
3336636991311,1cyclictest0-21swapper/320:50:013
3336636991311,1cyclictest0-21swapper/300:25:173
3336636991310,2cyclictest3383359-21sh19:55:013
333663399139,3cyclictest3513698-21kworker/u8:2+flush-259:022:35:262
333663399139,2cyclictest500-21systemd-logind22:05:012
333663399138,4cyclictest3352456-21store19:25:032
3336633991310,2cyclictest3626844-21ntp_kernel_pll_23:45:172
3336633991310,2cyclictest3616952-21tail23:35:272
3336633991310,2cyclictest3441230-21irqcore20:50:142
333663199138,3cyclictest3456433-21cron21:04:591
3336631991310,2cyclictest3362479-21kworker/u8:3+events_unbound20:10:141
3336631991310,2cyclictest3357268-21kworker/u8:1-flush-259:020:10:001
3336630991310,2cyclictest3519255-21cpuspeed_turbos22:05:120
3336630991310,2cyclictest3357268-21kworker/u8:1+flush-259:020:17:440
333663699128,2cyclictest0-21swapper/323:10:013
333663699127,3cyclictest3539616-21latency_hist22:25:003
333663699126,4cyclictest3524448-21diskstats22:10:103
3336636991210,1cyclictest0-21swapper/319:25:013
333663399129,2cyclictest3665666-21tr00:25:132
333663399128,3cyclictest3482731-21kworker/u8:3+flush-259:021:45:262
333663399126,4cyclictest252-21systemd-journal23:20:142
333663399126,4cyclictest252-21systemd-journal21:54:552
3336633991210,1cyclictest0-21swapper/223:29:592
333663399120,4cyclictest3367840-21awk19:40:012
333663399120,10cyclictest1-21systemd00:06:122
333663199129,2cyclictest3498395-21kworker/u8:4+flush-259:022:20:281
333663199129,2cyclictest3398876-21kworker/u8:4+flush-259:021:08:181
333663199129,2cyclictest3336309-21kworker/u8:2+events_unbound19:15:211
333663199129,2cyclictest0-21swapper/121:55:131
333663199129,2cyclictest0-21swapper/121:55:121
333663199128,3cyclictest3513698-21kworker/u8:2+flush-259:023:05:151
333663199126,5cyclictest3627723-21idleruntime-cro23:50:001
333663199120,4cyclictest3597244-21grep23:20:171
333663699118,2cyclictest3627223-21unixbench_multi23:45:213
333663699118,2cyclictest3477873-21cpuspeed_turbos21:25:163
333663699118,2cyclictest0-21swapper/320:50:123
333663699117,3cyclictest252-21systemd-journal20:20:013
333663699116,3cyclictest34618692sleep321:10:133
333663399118,2cyclictest3529109-21kworker/u8:0-events_unbound23:00:162
333663399118,2cyclictest3342078-21cpuspeed_turbos19:15:142
333663399117,3cyclictest252-21systemd-journal20:20:062
333663399111,8cyclictest3659190-21awk00:20:002
333663399111,8cyclictest3659190-21awk00:19:592
333663399110,3cyclictest3466791-21latency_hist21:15:012
333663199119,1cyclictest3473073-21irqcore21:20:151
333663199118,2cyclictest3674254-21unixbench_multi00:30:211
333663199117,3cyclictest3440803-21apt-get20:50:101
333663699108,1cyclictest3544731-21munin-plugin-st22:30:013
333663699108,1cyclictest0-21swapper/323:35:213
333663699100,2cyclictest3424617-21cron20:35:013
333663399107,2cyclictest3440090-21unixbench_multi20:45:222
333663399107,2cyclictest3368221-21cpuspeed_turbos19:40:122
333663399106,2cyclictest3493045-21latency_hist21:40:012
333663399106,2cyclictest0-21swapper/200:30:092
333663399104,4cyclictest490-21dbus-daemon22:00:012
333663399104,4cyclictest490-21dbus-daemon22:00:002
333663399101,7cyclictest3524034-21basename22:09:592
333663399100,4cyclictest3534405-21munin-run22:20:002
33366369997,1cyclictest3493680-21mii-tool21:40:163
33366369995,3cyclictest3374052-21irqcore19:45:173
33366339995,2cyclictest3569730-21ntpq22:50:172
33366339993,4cyclictest3456718-21apt-get21:05:122
33366339990,7cyclictest3544724-21latency_hist22:30:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional