You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-13 - 16:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sat Dec 13, 2025 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2908003994135,4cyclictest2969956-21latency_hist08:10:002
2908000993631,4cyclictest252-21systemd-journal07:20:011
2908000993528,5cyclictest3063162-21latency_hist09:40:011
2908003993125,4cyclictest3068753-21diskstats09:45:152
2908000993123,5cyclictest3125043-21latency_hist10:40:001
2907995992925,3cyclictest3037408-21kworker/u8:1+flush-259:009:35:200
2907995992825,2cyclictest2902434-21kworker/u8:4+events_unbound07:40:240
2907995992724,2cyclictest3037408-21kworker/u8:1+events_unbound09:20:220
2907995992720,5cyclictest2959631-21latency_hist08:00:010
2908003992623,2cyclictest3151725-21kworker/u8:2+flush-259:011:05:182
2908000992619,5cyclictest509-21rs:main1
2908000992619,5cyclictest3119902-21latency_hist10:35:001
2907995992623,2cyclictest3016792-21kworker/u8:3+flush-259:009:15:210
2907995992617,6cyclictest252-21systemd-journal10:05:000
2908000992521,3cyclictest3109849-21apt-get10:25:151
2907995992519,4cyclictest3172756-21kworker/u8:4+flush-259:011:30:010
2907995992519,4cyclictest3172756-21kworker/u8:4+flush-259:011:30:000
2908000992418,4cyclictest3047764-21latency_hist09:25:011
2908000992417,4cyclictest2913100-21latency_hist07:15:011
2907995992421,2cyclictest3239990-21kworker/u8:4+flush-259:012:40:000
2908003992319,3cyclictest3006761-21apt-get08:45:142
2908000992319,3cyclictest3239784-21kworker/u8:3+flush-259:012:30:181
2907995992320,2cyclictest3078135-21kworker/u8:1+flush-259:010:15:000
2907995992319,3cyclictest2943698-21kworker/u8:2+events_unbound07:45:150
2908006992217,4cyclictest3016769-21latency_hist08:55:013
2908000992217,4cyclictest2943698-21kworker/u8:2+flush-259:007:45:171
2908000992216,4cyclictest252-21systemd-journal10:05:311
2908000992215,5cyclictest509-21rs:main1
2907995992219,2cyclictest3114766-21kworker/u8:0+events_unbound10:40:180
2907995992219,2cyclictest3016792-21kworker/u8:3+flush-259:009:10:000
290799599221,19cyclictest858-21sshd10:50:390
2908000992116,3cyclictest252-21systemd-journal12:35:581
2907995992117,3cyclictest2996019-21kworker/u8:1+flush-259:008:50:000
2907995992117,3cyclictest2902434-21kworker/u8:4+flush-259:007:20:180
2907995992114,5cyclictest3052854-21latency_hist09:30:010
2908003992015,4cyclictest3078135-21kworker/u8:1+flush-259:010:25:182
290800099205,6cyclictest3135395-21munin-run10:50:001
2908000992015,3cyclictest252-21systemd-journal07:36:141
2908000992014,4cyclictest3006732-21/usr/sbin/munin08:45:161
2907995992017,2cyclictest3145863-21kworker/u8:1+events_unbound12:25:200
2907995992017,2cyclictest3099275-21kworker/u8:2+events_unbound10:30:250
2907995992017,2cyclictest2907975-21kworker/u8:0+flush-259:007:25:260
2907995992016,3cyclictest3145863-21kworker/u8:1+flush-259:011:20:000
2907995992014,4cyclictest3146608-21cat11:00:000
2908006991914,4cyclictest252-21systemd-journal12:17:003
2908000991915,3cyclictest252-21systemd-journal12:04:551
2908000991914,4cyclictest2902434-21kworker/u8:4+flush-259:007:25:011
2907995991916,2cyclictest2949340-21kworker/u8:0+flush-259:008:10:160
2907995991915,3cyclictest3145863-21kworker/u8:1+flush-259:012:20:210
2908006991815,2cyclictest3203686-21kworker/u8:2+flush-259:012:00:263
2908006991815,2cyclictest3078135-21kworker/u8:1+flush-259:010:30:003
2908006991812,4cyclictest3104685-21sshd10:20:133
2908003991816,1cyclictest0-21swapper/207:15:142
2908003991815,2cyclictest3078135-21kworker/u8:1+events_unbound10:15:242
2908000991814,2cyclictest0-21swapper/111:20:001
2908000991813,3cyclictest3125090-21kworker/u8:3+flush-259:010:50:181
2908000991812,4cyclictest2980764-21diskstats08:20:141
290800099180,17cyclictest1-21systemd12:05:511
2907995991815,2cyclictest3042811-21kworker/u8:4+events_unbound09:45:240
2907995991814,3cyclictest3016792-21kworker/u8:3+events_unbound09:13:140
2907995991813,4cyclictest2996019-21kworker/u8:1+flush-259:008:50:190
2908006991715,1cyclictest2913783-21irqcore07:15:133
2908006991714,2cyclictest2954680-21kworker/u8:1+flush-259:008:00:253
2908006991713,3cyclictest2918564-21/usr/sbin/munin07:20:153
2908003991713,3cyclictest3053122-21apt-get09:30:122
2908003991711,4cyclictest2958804-21ntpq07:55:192
2908000991711,4cyclictest3145863-21kworker/u8:1+flush-259:012:10:161
2907995991714,2cyclictest60350irq/132-enp0s29f1:rx-009:55:230
2907995991714,2cyclictest60350irq/132-enp0s29f1:rx-007:35:230
2907995991714,2cyclictest2969980-21kworker/u8:2+flush-259:008:20:240
2907995991713,3cyclictest3188127-21kworker/u8:0+events_unbound11:50:220
2907995991713,2cyclictest60350irq/132-enp0s29f1:rx-008:00:240
2907995991712,3cyclictest2954454-21turbostat07:55:000
2907995991710,5cyclictest3162015-21cron11:15:000
2907995991710,5cyclictest3000950-21latency_hist08:40:000
2908003991613,2cyclictest3033572-21kthreadcore09:10:352
2908003991613,2cyclictest2933833-21sh07:35:012
2908003991613,2cyclictest252-21systemd-journal12:35:002
2908003991613,2cyclictest0-21swapper/208:05:012
2908003991612,3cyclictest3146087-21apt-get11:00:012
2907995991614,1cyclictest60350irq/132-enp0s29f1:rx-012:15:160
2907995991614,1cyclictest60350irq/132-enp0s29f1:rx-012:10:060
2907995991613,2cyclictest60350irq/132-enp0s29f1:rx-011:45:170
2907995991613,2cyclictest60350irq/132-enp0s29f1:rx-011:05:140
2907995991613,2cyclictest60350irq/132-enp0s29f1:rx-010:35:210
2907995991613,2cyclictest60350irq/132-enp0s29f1:rx-010:15:170
2907995991613,2cyclictest60350irq/132-enp0s29f1:rx-008:31:520
2907995991613,2cyclictest2907975-21kworker/u8:0+flush-259:007:15:110
2907995991612,3cyclictest60350irq/132-enp0s29f1:rx-012:05:140
2907995991612,3cyclictest60350irq/132-enp0s29f1:rx-011:30:220
2907995991612,3cyclictest3078135-21kworker/u8:1+flush-259:010:20:240
2908006991512,2cyclictest3152068-21cpuspeed_turbos11:05:133
2908006991511,3cyclictest252-21systemd-journal09:30:003
290800399159,4cyclictest2954451-21latency_hist07:55:012
2908003991513,1cyclictest3193350-21turbostat.cron11:45:012
2908003991513,1cyclictest2923511-21idleruntime-cro07:25:012
2908003991513,1cyclictest0-21swapper/208:45:002
2908003991511,3cyclictest3203686-21kworker/u8:2+flush-259:011:55:192
2908003991510,3cyclictest509-21rs:main2
290800099158,5cyclictest3083812-21latency_hist10:00:011
2908000991512,2cyclictest2943698-21kworker/u8:2+flush-259:007:55:111
2908000991511,3cyclictest503-21tpm2-abrmd11:35:001
2908000991511,3cyclictest2970402-21fschecks_count08:10:151
2908000991510,3cyclictest509-21rs:main1
2907995991513,1cyclictest60350irq/132-enp0s29f1:rx-010:57:130
2907995991513,1cyclictest0-21swapper/012:00:190
2907995991512,2cyclictest61150irq/133-enp0s29f1:tx-009:40:350
2907995991512,2cyclictest60350irq/132-enp0s29f1:rx-012:34:070
2907995991512,2cyclictest60350irq/132-enp0s29f1:rx-011:40:210
2907995991512,2cyclictest60350irq/132-enp0s29f1:rx-010:45:140
2907995991512,2cyclictest60350irq/132-enp0s29f1:rx-009:50:160
2907995991512,2cyclictest60350irq/132-enp0s29f1:rx-009:00:210
2907995991512,2cyclictest3083826-21kworker/u8:0+flush-259:010:05:210
2907995991512,1cyclictest0-21swapper/011:35:140
2907995991511,3cyclictest2996019-21kworker/u8:1+events_unbound08:40:330
290800699148,4cyclictest3129302-21ntpq10:40:243
2908006991412,1cyclictest0-21swapper/310:50:003
2908006991411,2cyclictest3125053-21idleruntime-cro10:39:593
2908006991410,3cyclictest3099275-21kworker/u8:2+flush-259:010:30:193
290800699140,13cyclictest1-21systemd10:18:133
2908003991410,3cyclictest3203686-21kworker/u8:2+flush-259:012:00:162
2908003991410,3cyclictest3016792-21kworker/u8:3+flush-259:009:15:262
290800099147,5cyclictest3067518-21ntpq09:40:421
2908000991411,2cyclictest3038092-21irqcore09:15:171
2908000991411,2cyclictest2996019-21kworker/u8:1+flush-259:008:55:261
2908000991411,2cyclictest2980353-21idleruntime-cro08:20:001
2908000991410,3cyclictest3016766-21idleruntime-cro08:55:001
2908000991410,3cyclictest2963866-21ntp_kernel_err08:00:201
290799599148,5cyclictest252-21systemd-journal11:20:000
290799599148,5cyclictest252-21systemd-journal11:20:000
2907995991413,1cyclictest60350irq/132-enp0s29f1:rx-007:30:280
2907995991412,1cyclictest60350irq/132-enp0s29f1:rx-008:05:250
2907995991411,2cyclictest60350irq/132-enp0s29f1:rx-008:18:460
2907995991410,3cyclictest2902434-21kworker/u8:4+events_unbound07:10:190
290800699139,3cyclictest3006108-21kworker/u8:0+flush-259:008:45:183
290800699138,4cyclictest252-21systemd-journal09:15:003
290800699138,3cyclictest3078619-21sh09:55:003
2908006991310,2cyclictest3230172-21sed12:20:163
2908006991310,2cyclictest3098825-21unixbench_singl10:10:323
2908006991310,2cyclictest3068851-21fschecks_time09:45:163
2908006991310,2cyclictest2980670-21cpuspeed_turbos08:20:133
2908006991310,2cyclictest2970323-21cpuspeed_turbos08:10:143
2908006991310,2cyclictest2938579-21unixbench_singl07:35:273
2908006991310,2cyclictest2912676-21unixbench_singl07:10:213
290800399138,3cyclictest3245160-21/usr/sbin/munin12:35:162
2908003991311,1cyclictest0-21swapper/207:50:012
2908003991310,2cyclictest3168863-21kthreadcore11:20:172
2908003991310,2cyclictest3168863-21kthreadcore11:20:172
2908003991310,2cyclictest2940295-21kthreadcore07:40:182
290800099138,4cyclictest3234885-21apt-get12:25:131
2908000991311,1cyclictest0-21swapper/111:00:001
2907995991310,2cyclictest60350irq/132-enp0s29f1:rx-011:55:140
2907995991310,2cyclictest60350irq/132-enp0s29f1:rx-010:25:260
2907995991310,2cyclictest60350irq/132-enp0s29f1:rx-008:59:120
2907995991310,2cyclictest2969980-21kworker/u8:2+events_unbound08:25:240
290800699129,2cyclictest3057548-21unixbench_multi09:30:253
290800699129,2cyclictest1-21systemd08:34:533
290800699128,3cyclictest3218890-21sshd12:07:133
290800699128,3cyclictest3167811-21/usr/sbin/munin11:20:203
290800699128,3cyclictest3167811-21/usr/sbin/munin11:20:203
290800699128,3cyclictest3157110-21apt-get11:10:133
290800699128,3cyclictest3145863-21kworker/u8:1+flush-259:012:25:193
290800699127,3cyclictest3243996-21ntp_kernel_err12:30:203
2908006991210,1cyclictest3193590-21/usr/sbin/munin11:45:183
290800399129,2cyclictest3136489-21irqcore10:50:182
290800399128,3cyclictest3146961-21fschecks_count11:00:142
290800399128,3cyclictest3078620-21sh09:55:002
290800399128,2cyclictest3094095-21awk10:10:002
290800399128,2cyclictest3078135-21kworker/u8:1+flush-259:010:20:192
290800099129,2cyclictest3203980-21cpuspeed_turbos11:55:141
290800099129,2cyclictest3078163-21unixbench_multi09:50:251
290800099129,2cyclictest2965472-21sed08:05:171
290799599128,3cyclictest3052881-21kworker/u8:0+events_unbound09:34:570
290800699119,1cyclictest0-21swapper/310:05:003
290800699118,2cyclictest3187229-21needreboot11:35:203
290800699118,2cyclictest0-21swapper/312:10:253
290800699118,2cyclictest0-21swapper/311:40:213
290800699118,2cyclictest0-21swapper/311:00:003
290800699118,2cyclictest0-21swapper/308:45:003
290800699114,2cyclictest3063213-21cat09:40:013
290800699111,2cyclictest252-21systemd-journal09:25:003
290800399119,1cyclictest0-21swapper/210:00:302
290800399118,2cyclictest3229816-21cpuspeed_turbos12:20:142
290800399118,2cyclictest31153662sleep210:30:192
290800399118,2cyclictest3067885-21unixbench_multi09:40:452
290800399118,2cyclictest3022460-21fschecks_time09:00:152
290800399118,2cyclictest2980324-21cron08:20:002
290800399118,2cyclictest0-21swapper/208:35:222
290800399117,3cyclictest3198791-21apt-get11:50:122
290800399117,3cyclictest2985735-21apt-get08:25:132
290800399115,4cyclictest513-21gdbus10:50:002
290800399115,4cyclictest252-21systemd-journal10:38:132
290800399111,3cyclictest252-21systemd-journal10:15:002
290800099119,1cyclictest0-21swapper/111:25:141
290800099119,1cyclictest0-21swapper/111:25:141
290800099117,3cyclictest509-21rs:main1
290800099115,4cyclictest3001133-21cat08:40:001
290800699108,1cyclictest3150703-21needreboot11:00:193
290800699108,1cyclictest0-21swapper/310:50:003
290800699108,1cyclictest0-21swapper/309:05:203
290800699108,1cyclictest0-21swapper/307:50:173
290800699107,2cyclictest491ktimers/311:30:133
290800699104,4cyclictest3079045-21df09:55:163
290800399108,1cyclictest0-21swapper/208:10:162
290800399107,2cyclictest3239280-21unixbench_multi12:25:262
290800399107,2cyclictest3145863-21kworker/u8:1+flush-259:011:35:182
290800399107,1cyclictest2939014-21latency_hist07:40:012
290800399105,4cyclictest3219143-21basename12:10:012
290800399100,3cyclictest3032255-21latency_hist09:10:012
290800099108,1cyclictest0-21swapper/109:35:001
290800099107,2cyclictest2971-21cron12:25:011
290800099106,3cyclictest3125090-21kworker/u8:3+flush-259:011:38:201
29080069995,3cyclictest3166231-21ld11:15:213
29080069995,3cyclictest2965103-21awk08:05:133
29080069995,3cyclictest2927783-21ntpq07:25:223
29080039997,1cyclictest0-21swapper/211:30:182
29080009991,6cyclictest3202731-21ntpq11:50:221
29080009990,6cyclictest30326931cpuspeed_turbos09:10:311
29080069984,3cyclictest252-21systemd-journal07:55:133
29080069984,2cyclictest3024141-21kthreadcore09:00:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional