You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 22:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu Dec 25, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3773206993836,2cyclictest3997372-21kworker/u8:2+events_unbound11:05:142
3773206993633,2cyclictest3829253-21kworker/u8:4+events_unbound08:15:212
3773206993532,2cyclictest3819432-21kworker/u8:2+flush-259:008:05:222
3773211993429,3cyclictest3986021-21apt-get10:35:003
3773206993431,2cyclictest3949336-21kworker/u8:4+flush-259:010:10:232
3773206993431,2cyclictest3845356-21kworker/u8:3+events_unbound08:40:152
3773206993430,3cyclictest3740098-21kworker/u8:3+events_unbound07:10:212
3773200993431,2cyclictest3903139-21kworker/u8:1+flush-259:010:05:000
3773206993330,2cyclictest3773185-21kworker/u8:2+events_unbound07:40:212
3773206993330,2cyclictest3763558-21kworker/u8:1+events_unbound08:20:182
3773206993326,3cyclictest3997372-21kworker/u8:2+events_unbound11:05:002
3773202993324,8cyclictest75-21kswapd008:24:221
3773206993229,2cyclictest4048621-21kworker/u8:4+events_unbound11:35:222
3773211993127,2cyclictest3949337-21kworker/u8:5+flush-259:010:15:243
3773206993129,2cyclictest3845356-21kworker/u8:3+events_unbound09:10:242
3773206993128,2cyclictest3965134-21kworker/u8:0+events_unbound10:24:252
3773206993128,2cyclictest3897524-21kworker/u8:0+events_unbound09:35:232
3773206993127,3cyclictest3903139-21kworker/u8:1+flush-259:009:50:162
3773206993027,2cyclictest3845356-21kworker/u8:3+flush-259:008:35:212
3773206993027,2cyclictest3819432-21kworker/u8:2+events_unbound08:10:202
3773206993027,2cyclictest3763558-21kworker/u8:1+flush-259:007:35:162
3773206993026,3cyclictest4048621-21kworker/u8:4+flush-259:011:40:202
3773206993026,3cyclictest3903139-21kworker/u8:1+events_unbound09:15:222
3773200993025,4cyclictest3986145-21gpgv10:35:000
3773211992922,5cyclictest3850327-21latency_hist08:25:003
3773206992926,2cyclictest3912836-21kworker/u8:3+flush-259:009:30:212
3773206992926,2cyclictest3763558-21kworker/u8:1+events_unbound08:30:162
3773200992924,4cyclictest3860717-21latency_hist08:35:000
3773206992825,2cyclictest3763558-21kworker/u8:1+flush-259:008:00:162
3773206992824,3cyclictest3912836-21kworker/u8:3+events_unbound09:40:082
3773206992724,2cyclictest4048621-21kworker/u8:4+events_unbound11:55:222
3773206992724,2cyclictest3949336-21kworker/u8:4+flush-259:010:25:142
3773206992724,2cyclictest3763558-21kworker/u8:1+events_unbound08:55:242
3773206992723,3cyclictest4022655-21kworker/u8:1+events_unbound11:25:132
3773206992722,3cyclictest509-21rs:main2
3773202992723,3cyclictest3949336-21kworker/u8:4+flush-259:010:20:011
3773202992722,3cyclictest3908325-21/usr/sbin/munin09:20:201
3773200992724,2cyclictest3997372-21kworker/u8:2+events_unbound11:15:250
3773206992623,2cyclictest4105166-21kworker/u8:2+events_unbound12:35:192
3773206992623,2cyclictest3860740-21kworker/u8:2+events_unbound09:25:232
3773202992621,3cyclictest3891844-21gpgv09:05:001
3773200992623,2cyclictest4084828-21kworker/u8:0+flush-259:012:10:210
3773200992623,2cyclictest3897524-21kworker/u8:0+flush-259:009:40:210
3773200992622,3cyclictest3997372-21kworker/u8:2+flush-259:011:00:000
3773206992522,2cyclictest4094897-21kworker/u8:1+flush-259:012:30:252
3773206992522,2cyclictest3763558-21kworker/u8:1+events_unbound07:30:152
3773200992522,2cyclictest3829253-21kworker/u8:4+flush-259:008:25:160
3773200992521,3cyclictest509-21rs:main0
3773211992420,3cyclictest4089974-21/usr/sbin/munin12:15:153
3773211992418,4cyclictest4032910-21latency_hist11:20:003
3773206992421,2cyclictest3997372-21kworker/u8:2+events_unbound11:50:132
3773206992421,2cyclictest3991658-21kworker/u8:1+events_unbound10:40:182
3773206992421,2cyclictest3897524-21kworker/u8:0+flush-259:009:20:192
3773206992421,2cyclictest3845356-21kworker/u8:3+events_unbound08:50:242
3773206992420,3cyclictest4094897-21kworker/u8:1+events_unbound12:20:182
3773202992420,2cyclictest3975547-21apt-get10:25:101
3773200992419,4cyclictest3923568-21latency_hist09:35:010
3773211992320,2cyclictest3991658-21kworker/u8:1+flush-259:010:40:233
3773211992320,2cyclictest3829253-21kworker/u8:4+events_unbound08:25:223
3773211992319,3cyclictest3980563-21gpgv10:30:013
3773206992320,2cyclictest3763558-21kworker/u8:1+flush-259:007:20:152
3773206992320,2cyclictest3763558-21kworker/u8:1+events_unbound07:25:262
3773206992319,3cyclictest4007331-21kworker/u8:3+events_unbound11:10:232
3773206992319,3cyclictest3997372-21kworker/u8:2+flush-259:011:15:482
3773206992319,3cyclictest3949337-21kworker/u8:5+events_unbound10:51:192
3773206992317,5cyclictest3939031-21latency_hist09:50:012
3773202992320,2cyclictest3934105-21kworker/u8:2+events_unbound09:55:011
3773200992320,2cyclictest3829253-21kworker/u8:4+events_unbound08:15:230
3773200992319,3cyclictest3997372-21kworker/u8:2+flush-259:012:05:000
3773211992219,2cyclictest500-21systemd-logind07:15:013
3773206992219,2cyclictest3845356-21kworker/u8:3+events_unbound09:00:192
3773202992215,5cyclictest3981101-21kworker/u8:2+flush-259:010:30:181
3773200992219,2cyclictest4048621-21kworker/u8:4+flush-259:011:55:210
3773200992219,2cyclictest3897524-21kworker/u8:0+flush-259:009:20:160
3773200992219,2cyclictest3763558-21kworker/u8:1+flush-259:007:30:230
3773200992218,3cyclictest3965168-21apt-get10:15:110
3773200992218,2cyclictest4007118-21kworker/u8:1+flush-259:011:00:200
3773211992118,2cyclictest4105166-21kworker/u8:2+flush-259:012:30:273
3773211992118,2cyclictest3845356-21kworker/u8:3+flush-259:008:50:263
3773206992118,2cyclictest3997372-21kworker/u8:2+events_unbound10:45:172
3773206992118,2cyclictest3991657-21kworker/u8:0+flush-259:010:55:182
3773206992118,2cyclictest3949336-21kworker/u8:4+events_unbound10:05:222
3773206992118,2cyclictest3763558-21kworker/u8:1+events_unbound07:18:192
3773206992117,3cyclictest3981101-21kworker/u8:2+events_unbound10:35:122
3773202992116,3cyclictest4105390-21/usr/sbin/munin12:30:151
3773202992116,3cyclictest3991658-21kworker/u8:1+flush-259:010:40:161
3773200992118,3cyclictest3773185-21kworker/u8:2+events_unbound07:35:220
3773200992118,2cyclictest3949336-21kworker/u8:4+events_unbound10:25:010
377320099210,19cyclictest1-21systemd11:25:000
377320099210,19cyclictest1-21systemd11:25:000
3773211992017,2cyclictest3912836-21kworker/u8:3+events_unbound09:40:163
3773211992017,2cyclictest3897524-21kworker/u8:0+events_unbound09:25:173
3773211992017,2cyclictest3845356-21kworker/u8:3+flush-259:009:05:213
3773211992017,2cyclictest3763558-21kworker/u8:1+flush-259:007:30:203
3773211992016,3cyclictest3981101-21kworker/u8:2+flush-259:010:35:173
3773206992017,2cyclictest4043245-21kworker/u8:3+events_unbound11:31:182
3773206992017,2cyclictest3803531-21kworker/u8:3+events_unbound07:54:192
3773206992016,3cyclictest4022655-21kworker/u8:1+events_unbound11:20:192
3773206992016,3cyclictest4022655-21kworker/u8:1+events_unbound11:20:192
3773202992016,2cyclictest3997372-21kworker/u8:2+flush-259:011:50:171
3773202992014,5cyclictest252-21systemd-journal08:14:591
3773200992017,2cyclictest3763558-21kworker/u8:1+flush-259:007:55:200
3773211991916,2cyclictest4115464-21gpgv12:40:003
3773211991916,2cyclictest3997372-21kworker/u8:2+flush-259:011:50:263
3773211991916,2cyclictest3997372-21kworker/u8:2+events_unbound11:40:153
3773211991916,2cyclictest3997372-21kworker/u8:2+events_unbound11:25:203
3773211991916,2cyclictest3991658-21kworker/u8:1+events_unbound10:45:003
3773211991916,2cyclictest3949336-21kworker/u8:4+flush-259:010:10:233
3773211991915,3cyclictest509-21rs:main3
3773211991915,3cyclictest4032928-21kworker/u8:0+events_unbound11:30:183
3773211991915,3cyclictest3908351-21apt-get09:20:113
3773211991915,2cyclictest3845356-21kworker/u8:3+events_unbound08:35:173
3773211991914,4cyclictest3845132-21latency_hist08:20:003
3773206991916,2cyclictest3949337-21kworker/u8:5+events_unbound10:15:222
3773206991915,3cyclictest4064149-21kworker/u8:3+events_unbound12:00:032
3773202991913,4cyclictest4064149-21kworker/u8:3+flush-259:012:10:171
3773202991913,4cyclictest252-21systemd-journal11:43:181
3773200991917,2cyclictest4007331-21kworker/u8:3+flush-259:011:10:140
3773200991916,2cyclictest4064149-21kworker/u8:3+flush-259:012:15:200
3773200991916,2cyclictest3991657-21kworker/u8:0+events_unbound10:50:210
3773200991916,2cyclictest3991657-21kworker/u8:0+events_unbound10:46:190
3773200991916,2cyclictest3949336-21kworker/u8:4+events_unbound10:10:220
3773200991916,2cyclictest3845356-21kworker/u8:3+flush-259:008:40:210
3773200991916,2cyclictest3763558-21kworker/u8:1+flush-259:007:50:160
3773200991916,2cyclictest3763558-21kworker/u8:1+events_unbound07:25:150
3773200991916,2cyclictest3740098-21kworker/u8:3+flush-259:007:10:130
3773200991915,3cyclictest4094897-21kworker/u8:1+events_unbound12:30:220
377320099190,17cyclictest509-21rs:main0
3773211991816,2cyclictest3997372-21kworker/u8:2+flush-259:011:35:163
3773211991816,1cyclictest509-21rs:main3
3773211991816,1cyclictest0-21swapper/307:25:013
3773211991814,3cyclictest4007331-21kworker/u8:3+events_unbound11:05:203
3773211991814,3cyclictest3949336-21kworker/u8:4+flush-259:010:20:143
3773211991814,3cyclictest3773185-21kworker/u8:2+flush-259:007:35:203
3773211991814,2cyclictest3763558-21kworker/u8:1+flush-259:007:45:233
3773211991813,3cyclictest4094897-21kworker/u8:1+flush-259:012:25:173
3773206991816,1cyclictest4064149-21kworker/u8:3+events_unbound12:10:192
3773206991815,2cyclictest4094897-21kworker/u8:1+flush-259:012:25:162
3773206991813,3cyclictest4079698-21/usr/sbin/munin12:05:212
3773206991811,5cyclictest3824571-21latency_hist08:00:012
3773202991814,2cyclictest3809383-21apt-get07:45:131
3773202991813,3cyclictest3876414-21/usr/sbin/munin08:50:151
3773202991812,4cyclictest3897306-21apt-get09:10:001
3773202991812,4cyclictest252-21systemd-journal07:40:051
3773200991815,2cyclictest3897524-21kworker/u8:0+events_unbound09:25:150
3773200991814,3cyclictest4115553-21gpgv12:40:000
3773200991814,3cyclictest3860740-21kworker/u8:2+flush-259:008:45:130
3773200991814,3cyclictest3845356-21kworker/u8:3+flush-259:008:55:000
3773211991714,2cyclictest4027806-21idleruntime-cro11:15:003
3773206991714,2cyclictest3903139-21kworker/u8:1+flush-259:009:55:012
3773206991714,2cyclictest3845356-21kworker/u8:3+events_unbound08:29:192
3773202991714,2cyclictest4037112-21needreboot11:20:201
3773202991714,2cyclictest4037112-21needreboot11:20:191
3773202991713,3cyclictest1-21systemd07:15:001
3773200991714,2cyclictest3845356-21kworker/u8:3+events_unbound08:35:190
377320099170,15cyclictest507-21in:imuxsock09:19:590
3773211991613,2cyclictest3845356-21kworker/u8:3+flush-259:009:00:163
3773211991612,3cyclictest4064149-21kworker/u8:3+events_unbound12:05:203
3773211991612,3cyclictest3997372-21kworker/u8:2+flush-259:011:00:253
3773211991612,3cyclictest3903139-21kworker/u8:1+flush-259:009:45:163
3773206991613,2cyclictest3949337-21kworker/u8:5+events_unbound10:02:182
3773206991612,3cyclictest3860740-21kworker/u8:2+flush-259:009:05:302
3773202991613,2cyclictest3861388-21irqcore08:35:141
3773202991613,2cyclictest3840673-21irqcore08:15:171
3773200991614,1cyclictest60350irq/132-enp0s29f1:rx-009:58:350
3773200991613,2cyclictest60350irq/132-enp0s29f1:rx-012:05:210
3773200991613,2cyclictest3997372-21kworker/u8:2+events_unbound11:50:190
3773200991612,3cyclictest3773185-21kworker/u8:2+events_unbound07:40:180
3773200991612,2cyclictest0-21swapper/009:10:000
3773211991512,2cyclictest4058782-21kworker/u8:0+events_unbound11:45:173
3773211991512,2cyclictest3845356-21kworker/u8:3+flush-259:008:45:253
3773211991512,2cyclictest3763558-21kworker/u8:1+flush-259:008:00:203
3773211991511,3cyclictest4094897-21kworker/u8:1+events_unbound12:20:143
3773206991512,2cyclictest4084828-21kworker/u8:0+events_unbound12:15:382
377320299159,4cyclictest4002223-21apt-get10:50:111
3773202991512,2cyclictest3871706-21grep08:45:161
3773202991512,2cyclictest3819432-21kworker/u8:2+flush-259:008:07:241
3773202991511,3cyclictest509-21rs:main1
3773200991514,1cyclictest60350irq/132-enp0s29f1:rx-010:25:540
3773200991513,2cyclictest60350irq/132-enp0s29f1:rx-010:06:040
3773200991513,1cyclictest60350irq/132-enp0s29f1:rx-007:45:260
3773200991513,1cyclictest60350irq/132-enp0s29f1:rx-007:17:560
3773200991512,2cyclictest61150irq/133-enp0s29f1:tx-011:30:190
3773200991512,2cyclictest60350irq/132-enp0s29f1:rx-011:40:110
3773200991512,2cyclictest60350irq/132-enp0s29f1:rx-009:37:520
3773200991512,2cyclictest60350irq/132-enp0s29f1:rx-008:10:180
3773200991512,2cyclictest4094897-21kworker/u8:1+flush-259:012:20:250
3773200991511,3cyclictest3997372-21kworker/u8:2+events_unbound11:35:140
3773200991511,3cyclictest3845356-21kworker/u8:3+events_unbound09:00:140
3773211991411,2cyclictest4084828-21kworker/u8:0+flush-259:012:10:233
3773211991411,2cyclictest3949337-21kworker/u8:5+flush-259:010:50:233
3773211991411,2cyclictest3949337-21kworker/u8:5+events_unbound10:05:203
3773211991411,2cyclictest3871024-21sh08:44:593
3773211991411,2cyclictest3804316-21cpuspeed_turbos07:40:123
3773211991411,2cyclictest3763558-21kworker/u8:1+flush-259:007:50:183
3773211991410,2cyclictest3763558-21kworker/u8:1+events_unbound07:55:163
3773206991411,2cyclictest3997372-21kworker/u8:2+events_unbound11:47:212
3773206991411,2cyclictest3763558-21kworker/u8:1+flush-259:007:47:582
377320299143,3cyclictest507-21in:imuxsock10:05:001
3773202991412,1cyclictest0-21swapper/109:50:001
3773202991412,1cyclictest0-21swapper/109:00:001
3773202991411,2cyclictest3907124-21cut09:15:191
3773202991411,2cyclictest3780833-21kthreadcore07:15:181
377320299141,11cyclictest3899314-21sshd09:10:181
3773200991413,1cyclictest60350irq/132-enp0s29f1:rx-008:56:060
3773200991411,2cyclictest3845356-21kworker/u8:3+events_unbound09:10:280
3773200991410,3cyclictest4058782-21kworker/u8:0+flush-259:011:45:510
377321199136,5cyclictest252-21systemd-journal12:05:013
3773211991311,2cyclictest3763558-21kworker/u8:1+events_unbound08:55:223
3773211991310,2cyclictest3903139-21kworker/u8:1+events_unbound09:15:143
3773211991310,2cyclictest3740098-21kworker/u8:3+events_unbound07:15:183
377320299139,3cyclictest4064149-21kworker/u8:3+flush-259:012:27:241
377320299139,3cyclictest3763558-21kworker/u8:1+events_unbound07:20:191
377320299138,4cyclictest3997372-21kworker/u8:2+flush-259:012:05:151
377320299138,4cyclictest3763558-21kworker/u8:1+flush-259:007:32:491
377320299138,3cyclictest3814538-21apt-get07:50:121
3773202991310,2cyclictest4020221-21kthreadcore11:05:181
3773202991310,2cyclictest3962290-21kthreadcore10:10:191
3773202991310,1cyclictest4115445-21munin-run12:40:001
377320299130,5cyclictest4053608-21awk11:40:001
377320299130,4cyclictest39199491kthreadcore09:30:171
377320099139,3cyclictest60350irq/132-enp0s29f1:rx-009:50:170
3773200991312,1cyclictest60350irq/132-enp0s29f1:rx-008:05:220
3773200991311,1cyclictest60350irq/132-enp0s29f1:rx-009:47:330
3773200991311,1cyclictest60350irq/132-enp0s29f1:rx-008:20:570
3773200991310,2cyclictest60350irq/132-enp0s29f1:rx-007:22:450
3773200991310,2cyclictest4094897-21kworker/u8:1+flush-259:012:25:210
377321199129,2cyclictest3835395-21fschecks_time08:10:143
377321199129,2cyclictest3763558-21kworker/u8:1+flush-259:007:25:243
377321199128,3cyclictest3903139-21kworker/u8:1+flush-259:009:56:433
377321199128,3cyclictest252-21systemd-journal10:59:553
377320299129,2cyclictest4039350-21kthreadcore11:25:151
377320299129,1cyclictest0-21swapper/108:00:001
377320299128,3cyclictest3997372-21kworker/u8:2+flush-259:011:50:011
377320299127,4cyclictest3799129-21apt-get07:35:121
377320299127,4cyclictest0-21swapper/110:25:011
377320299126,4cyclictest3997062-21apt-get10:45:111
377320299126,4cyclictest3944418-21/usr/sbin/munin09:55:141
377320299121,9cyclictest4074291-21awk12:00:001
377320099128,3cyclictest60350irq/132-enp0s29f1:rx-010:35:110
377321199119,2cyclictest3845356-21kworker/u8:3+flush-259:008:30:153
377321199118,2cyclictest3903139-21kworker/u8:1+flush-259:009:50:393
377321199118,2cyclictest3902906-21sh09:15:003
377321199118,2cyclictest3829253-21kworker/u8:4+events_unbound08:05:173
377321199117,3cyclictest3949337-21kworker/u8:5+events_unbound10:00:133
377320299118,2cyclictest3855070-21unixbench_multi08:25:241
377320299118,2cyclictest3789859-21tr07:25:161
377321199107,2cyclictest3997372-21kworker/u8:2+flush-259:011:20:213
377321199107,2cyclictest3997372-21kworker/u8:2+flush-259:011:20:203
377320299108,1cyclictest0-21swapper/111:30:191
377320299107,2cyclictest4074762-21fschecks_count12:00:151
377320299107,1cyclictest0-21swapper/108:30:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional