You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-10 - 12:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Wed Dec 10, 2025 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3233590994441,2cyclictest3519133-21kworker/u8:4+events_unbound00:05:011
3233598994135,4cyclictest3280270-21latency_hist19:55:013
3233590994135,4cyclictest3384596-21latency_hist21:35:011
3233588993834,3cyclictest3228633-21kworker/u8:2+flush-259:019:40:220
3233588993731,4cyclictest3477727-21latency_hist23:05:020
3233593993631,4cyclictest150154-21kworker/u8:4+flush-259:019:35:172
3233590993633,2cyclictest3228633-21kworker/u8:2+events_unbound19:40:141
3233598993531,3cyclictest3291109-21gpgv20:05:003
3233590993531,3cyclictest3259739-21kworker/u8:0+flush-259:020:20:261
3233598993427,5cyclictest3535240-21diskstats00:00:133
3233590993431,2cyclictest3498348-21kworker/u8:2+flush-259:023:39:591
3233588993326,5cyclictest3316686-21ntpq20:25:230
3233598993227,3cyclictest3467058-21kworker/u8:1+flush-259:022:55:193
3233590993229,2cyclictest3259739-21kworker/u8:0+flush-259:019:50:181
3233588993229,2cyclictest3457328-21kworker/u8:2+events_unbound22:50:010
3233590993128,2cyclictest3498348-21kworker/u8:2+flush-259:000:15:231
3233590993128,2cyclictest3301737-21kworker/u8:3+events_unbound21:25:181
3233593993023,5cyclictest3560677-21latency_hist00:25:012
3233590993027,2cyclictest3436753-21kworker/u8:4+flush-259:022:45:001
3233590993027,2cyclictest3436753-21kworker/u8:4+events_unbound22:32:401
3233598992925,3cyclictest3379433-21kworker/u8:2+flush-259:022:05:013
3233598992924,4cyclictest3343549-21kworker/u8:4+flush-259:021:05:183
3233590992926,2cyclictest3555515-21kworker/u8:4+events_unbound00:20:181
3233590992926,2cyclictest3519133-21kworker/u8:4+flush-259:023:50:151
3233588992926,2cyclictest3259739-21kworker/u8:0+flush-259:020:05:170
3233588992922,5cyclictest2698871-21systemd-journal23:55:000
3233590992825,2cyclictest3301737-21kworker/u8:3+flush-259:020:25:261
3233590992825,2cyclictest3259739-21kworker/u8:0+events_unbound20:55:201
3233590992825,2cyclictest150154-21kworker/u8:4+flush-259:019:15:151
3233590992824,3cyclictest3498348-21kworker/u8:2+flush-259:000:15:001
3233588992825,2cyclictest3436753-21kworker/u8:4+flush-259:022:30:190
3233598992719,5cyclictest3467427-21latency_hist22:55:013
3233590992722,3cyclictest2698871-21systemd-journal22:05:001
3233588992725,1cyclictest3259739-21kworker/u8:0+events_unbound19:40:000
3233588992723,3cyclictest3436753-21kworker/u8:4+flush-259:022:40:010
3233588992723,3cyclictest3431385-21diskstats22:20:150
3233590992619,5cyclictest3399060-21ntpq21:45:191
3233588992622,3cyclictest3415693-21apt-get22:05:130
3233588992622,3cyclictest3301737-21kworker/u8:3+events_unbound21:32:420
3233588992621,4cyclictest3233503-21latency_hist19:10:000
3233593992522,2cyclictest3301737-21kworker/u8:3+events_unbound21:25:002
3233593992519,4cyclictest3285431-21latency_hist20:00:012
3233590992523,2cyclictest3228633-21kworker/u8:2+events_unbound20:05:281
3233590992522,2cyclictest3472790-21kworker/u8:4+flush-259:023:10:271
3233590992522,2cyclictest3457328-21kworker/u8:2+flush-259:022:50:161
3233590992522,2cyclictest3410315-21kworker/u8:0+events_unbound22:38:421
3233588992521,3cyclictest3301962-21apt-get20:15:160
3233598992420,3cyclictest3379433-21kworker/u8:2+flush-259:021:30:193
3233598992418,4cyclictest3285457-21apt-key20:00:003
3233598992417,5cyclictest3348487-21latency_hist21:00:003
3233593992420,3cyclictest3472790-21kworker/u8:4+events_unbound23:20:262
3233593992420,3cyclictest3410315-21kworker/u8:0+flush-259:022:00:172
3233593992418,4cyclictest3253738-21ntpq19:25:212
3233590992421,2cyclictest3259739-21kworker/u8:0+flush-259:021:20:151
3233588992421,2cyclictest3472790-21kworker/u8:4+flush-259:023:10:160
3233598992318,3cyclictest3228633-21kworker/u8:2+flush-259:019:40:153
3233593992318,3cyclictest3343549-21kworker/u8:4+events_unbound21:05:232
3233590992320,2cyclictest3301737-21kworker/u8:3+events_unbound21:06:591
3233590992320,2cyclictest3249676-21kworker/u8:1+flush-259:019:25:181
3233590992319,3cyclictest3343549-21kworker/u8:4+events_unbound21:00:191
3233588992320,2cyclictest3379433-21kworker/u8:2+events_unbound21:35:250
3233588992320,2cyclictest3228633-21kworker/u8:2+events_unbound19:20:210
3233588992318,4cyclictest3285949-21base6420:00:010
3233598992219,2cyclictest3301737-21kworker/u8:3+flush-259:021:45:003
3233593992219,2cyclictest3249676-21kworker/u8:1+events_unbound19:30:162
3233593992219,2cyclictest0-21swapper/221:29:592
3233593992218,3cyclictest3540185-21kworker/u8:3+events_unbound00:25:172
3233593992218,3cyclictest3524092-21apt-get23:50:002
3233593992217,4cyclictest3540185-21kworker/u8:3+flush-259:000:10:182
3233593992217,3cyclictest2698871-21systemd-journal20:15:012
3233593992216,4cyclictest3000063-21rs:main2
3233590992219,2cyclictest3415658-21kworker/u8:4+events_unbound22:05:191
3233590992219,2cyclictest3379433-21kworker/u8:2+events_unbound21:35:191
3233590992218,3cyclictest3540185-21kworker/u8:3+events_unbound00:32:441
3233588992218,3cyclictest3498348-21kworker/u8:2+events_unbound23:44:410
3233588992215,5cyclictest3507766-21ntpq23:30:220
3233598992116,3cyclictest3239183-21gpgv19:15:013
3233598992116,3cyclictest3239183-21gpgv19:15:003
3233598992115,4cyclictest3322978-21/usr/sbin/munin20:35:133
3233593992118,2cyclictest3472790-21kworker/u8:4+flush-259:023:10:192
3233593992118,2cyclictest3410315-21kworker/u8:0+flush-259:022:40:012
3233593992118,2cyclictest3410315-21kworker/u8:0+events_unbound22:20:002
3233593992116,4cyclictest3482885-21latency_hist23:10:012
3233593992115,5cyclictest3305995-21ntpq20:15:252
3233590992119,2cyclictest3259739-21kworker/u8:0+events_unbound20:46:541
3233590992118,2cyclictest3343549-21kworker/u8:4+flush-259:021:10:211
3233590992118,2cyclictest3280289-21kworker/u8:1+events_unbound20:00:261
3233590992118,2cyclictest3259739-21kworker/u8:0+events_unbound20:50:171
3233590992117,3cyclictest3565835-21kworker/u8:0+events_unbound00:35:191
3233590992117,3cyclictest3436215-21gpgv22:25:011
3233588992118,2cyclictest3259739-21kworker/u8:0+flush-259:020:10:270
3233588992117,3cyclictest3519133-21kworker/u8:4+events_unbound23:55:420
3233588992117,3cyclictest3519133-21kworker/u8:4+events_unbound23:55:410
3233588992116,3cyclictest3565835-21kworker/u8:0+events_unbound00:30:170
3233598992016,3cyclictest3379433-21kworker/u8:2+flush-259:022:00:003
3233593992017,2cyclictest3259739-21kworker/u8:0+flush-259:019:50:172
3233593992016,3cyclictest3566077-21apt-get00:30:122
3233593992016,3cyclictest3259739-21kworker/u8:0+events_unbound20:55:182
3233590992018,1cyclictest3259739-21kworker/u8:0+flush-259:020:35:171
3233590992017,2cyclictest3529940-21apt-get23:55:151
3233590992017,2cyclictest3529940-21apt-get23:55:141
3233590992017,2cyclictest3472790-21kworker/u8:4+flush-259:023:25:181
3233590992017,2cyclictest3472790-21kworker/u8:4+events_unbound23:01:411
3233590992017,2cyclictest3457328-21kworker/u8:2+events_unbound22:57:261
3233590992016,3cyclictest3524092-21apt-get23:50:001
3233590992014,5cyclictest3410263-21latency_hist22:00:011
3233588992017,2cyclictest3540185-21kworker/u8:3+events_unbound00:17:410
3233588992017,2cyclictest3482522-21kworker/u8:0+events_unbound23:15:150
3233588992017,2cyclictest3457328-21kworker/u8:2+events_unbound22:51:420
3233588992017,2cyclictest3301737-21kworker/u8:3+events_unbound21:15:190
3233588992017,2cyclictest3259739-21kworker/u8:0+flush-259:020:45:000
3233588992017,2cyclictest3000063-21rs:main0
3233588992016,3cyclictest3565835-21kworker/u8:0+events_unbound00:38:400
3233588992016,3cyclictest3555515-21kworker/u8:4+events_unbound00:20:250
3233588992016,3cyclictest3436753-21kworker/u8:4+events_unbound22:25:140
3233598991915,3cyclictest3410315-21kworker/u8:0+flush-259:022:20:013
3233598991915,3cyclictest3259739-21kworker/u8:0+flush-259:020:35:003
3233598991913,4cyclictest3000063-21rs:main3
3233593991916,2cyclictest3534840-21kworker/u8:0+events_unbound00:00:202
3233593991915,3cyclictest3436753-21kworker/u8:4+flush-259:022:30:262
3233593991915,3cyclictest3379433-21kworker/u8:2+flush-259:022:20:152
3233593991914,4cyclictest3410315-21kworker/u8:0+events_unbound22:10:182
3233590991917,2cyclictest3467058-21kworker/u8:1+events_unbound23:05:241
3233590991916,2cyclictest3555515-21kworker/u8:4+events_unbound00:26:411
3233590991916,2cyclictest3482522-21kworker/u8:0+events_unbound23:15:191
3233588991916,2cyclictest3228633-21kworker/u8:2+events_unbound19:31:400
3233588991915,3cyclictest3508637-21kworker/u8:3+events_unbound23:38:410
3233588991915,3cyclictest3301737-21kworker/u8:3+events_unbound20:39:400
3233588991914,2cyclictest3301737-21kworker/u8:3+events_unbound21:55:010
3233588991913,4cyclictest3338154-21latency_hist20:50:010
3233588991913,4cyclictest3280692-21diskstats19:55:130
3233598991812,4cyclictest3436617-21cat22:25:023
3233593991814,2cyclictest3518925-21kworker/u8:1+events_unbound23:47:342
3233590991816,2cyclictest3301737-21kworker/u8:3+events_unbound21:50:221
3233590991815,2cyclictest3457328-21kworker/u8:2+events_unbound22:47:171
3233590991815,2cyclictest3259739-21kworker/u8:0+flush-259:020:30:241
3233588991816,2cyclictest3518925-21kworker/u8:1+events_unbound23:45:140
3233588991815,3cyclictest3379433-21kworker/u8:2+flush-259:021:55:260
3233588991815,2cyclictest3343549-21kworker/u8:4+events_unbound21:00:170
3233588991815,2cyclictest3259739-21kworker/u8:0+flush-259:021:20:190
3233588991814,3cyclictest3353666-21kworker/u8:2+events_unbound21:10:200
3233598991715,1cyclictest0-21swapper/322:25:003
3233593991715,1cyclictest3249439-21idleruntime-cro19:25:002
3233593991714,2cyclictest3498348-21kworker/u8:2+events_unbound00:05:252
3233593991714,2cyclictest3395090-21apt-get21:45:122
3233593991714,2cyclictest2698871-21systemd-journal20:40:012
3233590991714,2cyclictest3498348-21kworker/u8:2+events_unbound23:40:211
3233590991714,2cyclictest3488066-21kworker/u8:1+flush-259:023:30:251
3233590991714,2cyclictest3301737-21kworker/u8:3+events_unbound21:15:281
3233590991713,3cyclictest3228633-21kworker/u8:2+flush-259:019:10:151
3233590991713,3cyclictest3228633-21kworker/u8:2+flush-259:019:10:151
3233588991715,2cyclictest3472790-21kworker/u8:4+events_unbound23:20:270
3233588991714,2cyclictest3467058-21kworker/u8:1+events_unbound23:05:210
3233588991712,4cyclictest3472790-21kworker/u8:4+events_unbound23:25:140
3233598991613,2cyclictest3448108-21kthreadcore22:35:203
3233598991613,2cyclictest3259739-21kworker/u8:0-ext4-rsv-conversion20:20:003
3233598991613,2cyclictest0-21swapper/323:14:593
3233598991612,3cyclictest3473839-21kthreadcore23:00:173
3233598991612,3cyclictest0-21swapper/323:54:593
3233598991610,4cyclictest3249444-21latency_hist19:25:003
323359399163,8cyclictest3410272-21munin-run22:00:012
3233593991613,2cyclictest3513786-21sh23:39:592
3233593991613,2cyclictest3472790-21kworker/u8:4+events_unbound23:25:212
3233593991612,3cyclictest3436764-21/usr/sbin/munin22:25:172
3233593991611,4cyclictest2698871-21systemd-journal20:35:002
3233590991614,2cyclictest3498348-21kworker/u8:2+events_unbound00:05:231
3233590991613,2cyclictest3410315-21kworker/u8:0+flush-259:022:10:241
3233590991613,2cyclictest150154-21kworker/u8:4+events_unbound19:35:281
3233590991613,1cyclictest0-21swapper/120:20:011
3233588991614,1cyclictest60650irq/133-enp0s29f1:rx-022:00:430
3233588991614,1cyclictest60650irq/133-enp0s29f1:rx-019:45:240
3233588991613,2cyclictest60650irq/133-enp0s29f1:rx-021:25:270
3233588991613,2cyclictest60650irq/133-enp0s29f1:rx-019:50:430
3233588991613,2cyclictest3239855-21irqcore19:15:180
3233588991612,3cyclictest3540185-21kworker/u8:3+flush-259:000:10:200
3233588991612,3cyclictest3259739-21kworker/u8:0+events_unbound21:45:130
3233588991612,3cyclictest3259739-21kworker/u8:0+events_unbound20:32:410
3233588991611,4cyclictest3565819-21latency_hist00:30:010
3233598991511,3cyclictest2698871-21systemd-journal22:41:413
3233598991510,3cyclictest3259739-21kworker/u8:0+flush-259:020:40:263
3233593991512,2cyclictest3571659-21irqcore00:35:172
3233593991512,2cyclictest3394863-21kworker/u8:1+flush-259:022:10:002
3233593991512,2cyclictest3379433-21kworker/u8:2+events_unbound21:30:152
3233593991512,2cyclictest3228633-21kworker/u8:2+flush-259:019:40:232
3233593991511,3cyclictest3301737-21kworker/u8:3+events_unbound21:50:162
3233588991513,2cyclictest3498348-21kworker/u8:2+events_unbound00:05:310
3233588991513,2cyclictest3259739-21kworker/u8:0+events_unbound20:55:160
3233588991513,1cyclictest60650irq/133-enp0s29f1:rx-021:43:000
3233588991513,1cyclictest60650irq/133-enp0s29f1:rx-019:10:160
3233588991513,1cyclictest60650irq/133-enp0s29f1:rx-019:10:160
3233588991512,2cyclictest60650irq/133-enp0s29f1:rx-022:10:150
3233598991411,2cyclictest3481956-21ntp_kernel_err23:05:213
3233598991411,2cyclictest3379414-21latency_hist21:29:593
3233598991410,2cyclictest0-21swapper/322:50:003
323359899140,5cyclictest32943442kthreadcore20:05:193
3233593991411,2cyclictest3259739-21kworker/u8:0+flush-259:020:50:172
3233593991410,3cyclictest3519133-21kworker/u8:4+events_unbound23:55:242
3233593991410,3cyclictest3519133-21kworker/u8:4+events_unbound23:55:242
3233593991410,3cyclictest3228633-21kworker/u8:2+events_unbound19:45:232
3233590991411,2cyclictest3228633-21kworker/u8:2+events_unbound19:30:111
3233588991411,2cyclictest60650irq/133-enp0s29f1:rx-020:20:150
3233588991411,2cyclictest3259739-21kworker/u8:0+flush-259:020:50:210
3233588991410,3cyclictest3467058-21kworker/u8:1+flush-259:022:55:200
323359899138,3cyclictest3544219-21ntpq00:05:253
323359899133,1cyclictest0-21swapper/323:40:143
323359899131,3cyclictest3317576-21awk20:30:013
323359399139,3cyclictest3467944-21fschecks_time22:55:162
323359399139,2cyclictest0-21swapper/220:05:002
323359399138,3cyclictest2698871-21systemd-journal20:29:552
3233593991310,2cyclictest3343549-21kworker/u8:4+events_unbound21:10:252
3233590991310,2cyclictest3379433-21kworker/u8:2+flush-259:022:25:201
3233590991310,2cyclictest3259739-21kworker/u8:0+flush-259:021:42:061
3233588991311,1cyclictest60650irq/133-enp0s29f1:rx-022:43:410
3233588991311,1cyclictest0-21swapper/019:30:010
323359899129,2cyclictest3563224-21kthreadcore00:25:203
323359899129,2cyclictest3519133-21kworker/u8:4+flush-259:023:55:283
323359899129,2cyclictest3519133-21kworker/u8:4+flush-259:023:55:283
323359899129,2cyclictest3360093-21tr21:10:193
323359899129,1cyclictest0-21swapper/300:15:003
323359899128,3cyclictest3249676-21kworker/u8:1+flush-259:019:25:163
323359899127,4cyclictest3218051-21kworker/u8:0+flush-259:019:15:263
323359899127,3cyclictest2698871-21systemd-journal22:07:083
323359899121,9cyclictest3259713-21awk19:35:003
323359399128,3cyclictest3472790-21kworker/u8:4+events_unbound23:00:132
323359399128,3cyclictest3457328-21kworker/u8:2+flush-259:022:45:242
323359399128,3cyclictest3457328-21kworker/u8:2+events_unbound22:50:132
323359399128,3cyclictest3301737-21kworker/u8:3+flush-259:021:15:262
323359399128,3cyclictest3228633-21kworker/u8:2+flush-259:020:08:452
3233593991210,1cyclictest0-21swapper/219:10:172
3233593991210,1cyclictest0-21swapper/219:10:172
323359099128,3cyclictest3297019-21fschecks_count20:10:171
3233590991210,2cyclictest3259739-21kworker/u8:0+flush-259:020:40:211
3233590991210,2cyclictest3228633-21kworker/u8:2+events_unbound19:20:231
323359899118,2cyclictest3493497-21cpuspeed_turbos23:20:153
323359899118,2cyclictest3351043-21kthreadcore21:00:193
323359899118,2cyclictest3333351-21cpuspeed_turbos20:45:143
323359899118,2cyclictest0-21swapper/323:50:013
323359899117,3cyclictest3498348-21kworker/u8:2+flush-259:023:35:263
323359399117,3cyclictest3550563-21apt-get00:15:122
323359399111,8cyclictest3353635-21awk21:04:592
323358899118,2cyclictest3301737-21kworker/u8:3+events_unbound22:17:410
323359899108,1cyclictest0-21swapper/319:40:003
323359899107,2cyclictest3404722-21unixbench_singl21:50:253
323359899107,2cyclictest3307635-21cpuspeed_turbos20:20:133
323359899105,4cyclictest3421058-21diskstats22:10:143
323359899100,3cyclictest3343321-21latency_hist20:55:013
323359399105,4cyclictest3307515-21cat20:20:012
323359099106,3cyclictest3228633-21kworker/u8:2+events_unbound19:47:421
32335989995,3cyclictest3498312-21cron23:29:543
32335989995,3cyclictest3369670-21tune2fs21:20:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional