You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 04:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Tue Dec 09, 2025 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27406995147,3cyclictest75-21kswapd021:35:032
27406995147,3cyclictest75-21kswapd021:35:022
27410994339,3cyclictest183883-21kworker/u8:0+events_unbound21:50:013
27410994037,2cyclictest230430-21kworker/u8:4+events_unbound22:30:173
27406994037,2cyclictest75-21kswapd021:33:002
27410993936,2cyclictest48408-21kworker/u8:0+events_unbound19:35:163
27410993633,2cyclictest183883-21kworker/u8:0+flush-259:021:50:153
27410993532,2cyclictest27386-21kworker/u8:1+events_unbound19:10:253
27410993532,2cyclictest100594-21kworker/u8:4+events_unbound20:25:173
27404993532,2cyclictest74623-21kworker/u8:2+flush-259:020:00:181
27410993431,2cyclictest116296-21kworker/u8:0+flush-259:020:55:243
27410993330,2cyclictest266623-21kworker/u8:3+events_unbound23:00:153
27410993330,2cyclictest214854-21kworker/u8:3+flush-259:022:15:153
27410993329,3cyclictest99457-21kworker/u8:3+events_unbound21:35:023
27410993329,3cyclictest99457-21kworker/u8:3+events_unbound21:35:013
27406993329,3cyclictest199156-21kworker/u8:1+flush-259:022:45:002
27399993327,4cyclictest240578-21kworker/u8:2+flush-259:023:05:170
27410993229,2cyclictest199156-21kworker/u8:1+flush-259:022:45:233
27406993228,3cyclictest4183905-21kworker/u8:0+events_unbound19:10:002
27399993227,4cyclictest317591-21ntpq23:45:200
27410993128,2cyclictest59116-21kworker/u8:1+events_unbound20:35:003
27410993128,2cyclictest282104-21kworker/u8:4+events_unbound23:29:593
27410993128,2cyclictest266623-21kworker/u8:3+flush-259:023:20:153
27410993026,3cyclictest59116-21kworker/u8:1+flush-259:019:40:173
27410993026,3cyclictest308481-21/usr/sbin/munin23:40:233
27406993027,2cyclictest48408-21kworker/u8:0+events_unbound19:40:002
27406993023,5cyclictest2698871-21systemd-journal21:14:592
27404993025,3cyclictest276761-21kworker/u8:0+flush-259:023:50:001
27399993015,13cyclictest69653-21/usr/sbin/munin19:50:140
27410992927,1cyclictest225041-21kworker/u8:2+flush-259:022:20:243
27410992926,2cyclictest99457-21kworker/u8:3+events_unbound21:00:163
27410992926,2cyclictest297660-21kworker/u8:2+events_unbound23:30:253
27410992926,2cyclictest282104-21kworker/u8:4+flush-259:023:15:273
27410992926,2cyclictest276761-21kworker/u8:0+flush-259:023:55:243
27410992923,4cyclictest2698871-21systemd-journal00:10:003
27406992925,3cyclictest116296-21kworker/u8:0+flush-259:020:40:252
27410992825,2cyclictest38321-21kworker/u8:4+events_unbound19:45:223
27410992825,2cyclictest240784-21kworker/u8:3+flush-259:022:35:163
27410992825,2cyclictest199156-21kworker/u8:1+events_unbound22:00:243
27410992823,2cyclictest99457-21kworker/u8:3+events_unbound21:34:293
27404992822,4cyclictest59332-21/usr/sbin/munin19:40:161
27399992821,5cyclictest167739-21latency_hist21:25:000
27399992814,12cyclictest0-21swapper/020:54:590
27410992724,2cyclictest4157673-21kworker/u8:3+events_unbound19:15:193
27410992723,3cyclictest276720-21latency_hist23:10:013
27404992723,3cyclictest27386-21kworker/u8:1+flush-259:019:10:201
27404992719,6cyclictest37853-21cp19:20:001
27399992711,5cyclictest178116-21apt-get21:35:010
27399992711,5cyclictest178116-21apt-get21:35:000
27410992623,2cyclictest240578-21kworker/u8:2+events_unbound22:50:183
27410992623,2cyclictest199156-21kworker/u8:1+events_unbound21:55:153
27410992622,3cyclictest59116-21kworker/u8:1+events_unbound20:50:183
27406992623,2cyclictest99457-21kworker/u8:3+events_unbound20:21:282
27410992522,2cyclictest59116-21kworker/u8:1+events_unbound21:25:143
27410992522,2cyclictest152247-21kworker/u8:4+flush-259:021:40:253
27410992522,2cyclictest152247-21kworker/u8:4+events_unbound21:20:253
27406992522,2cyclictest297660-21kworker/u8:2+flush-259:023:35:142
27406992521,3cyclictest214854-21kworker/u8:3+events_unbound22:17:252
27406992521,3cyclictest199156-21kworker/u8:1+events_unbound22:10:152
27406992521,3cyclictest152247-21kworker/u8:4+events_unbound22:08:272
27406992520,3cyclictest3000063-21rs:main2
27406992520,3cyclictest2698871-21systemd-journal00:00:002
27406992519,5cyclictest318760-21/usr/sbin/munin23:50:172
27406992519,5cyclictest318760-21/usr/sbin/munin23:50:162
27410992421,2cyclictest99457-21kworker/u8:3+flush-259:020:20:193
27410992421,2cyclictest99457-21kworker/u8:3+events_unbound21:05:163
27410992421,2cyclictest360420-21kworker/u8:0+events_unbound00:30:173
27410992421,2cyclictest339207-21kworker/u8:0+flush-259:000:15:263
27410992420,3cyclictest297660-21kworker/u8:2+flush-259:023:35:243
27410992420,3cyclictest276986-21apt-get23:10:123
27406992421,2cyclictest79944-21kworker/u8:3+events_unbound20:01:592
27406992417,6cyclictest349894-21latency_hist00:20:012
27404992421,2cyclictest503-21systemd-logind23:55:001
27404992421,2cyclictest503-21systemd-logind23:55:001
27404992419,4cyclictest38320-21kworker/u8:2+flush-259:019:20:161
27404992419,4cyclictest38320-21kworker/u8:2+flush-259:019:20:161
27404992417,5cyclictest157408-21latency_hist21:15:001
27410992320,2cyclictest90076-21kworker/u8:0+events_unbound20:15:233
27410992320,2cyclictest79944-21kworker/u8:3+events_unbound20:05:013
27410992320,2cyclictest38320-21kworker/u8:2+events_unbound19:20:173
27410992320,2cyclictest38320-21kworker/u8:2+events_unbound19:20:173
27410992319,3cyclictest329063-21kworker/u8:4+events_unbound00:10:263
27410992316,5cyclictest329066-21/usr/sbin/munin00:00:143
27406992320,3cyclictest152247-21kworker/u8:4+events_unbound21:22:162
27406992319,3cyclictest99457-21kworker/u8:3+events_unbound20:50:002
27406992319,3cyclictest59116-21kworker/u8:1+events_unbound20:50:172
27406992319,3cyclictest276761-21kworker/u8:0+events_unbound23:40:172
27406992318,3cyclictest281875-21latency_hist23:15:002
27404992316,5cyclictest79721-21latency_hist20:00:021
27410992219,2cyclictest74623-21kworker/u8:2+events_unbound20:10:173
27406992218,3cyclictest74623-21kworker/u8:2+events_unbound20:07:192
27406992218,3cyclictest282104-21kworker/u8:4+flush-259:023:25:002
27406992218,3cyclictest137118-21apt-get20:55:162
27406992217,4cyclictest183883-21kworker/u8:0+flush-259:022:20:192
2740499220,20cyclictest256072-21munin-plugin-st22:50:011
27399992217,3cyclictest100594-21kworker/u8:4+events_unbound20:25:200
27399992215,5cyclictest214617-21latency_hist22:10:010
27410992118,2cyclictest183883-21kworker/u8:0+events_unbound22:25:263
27410992117,3cyclictest199156-21kworker/u8:1+events_unbound22:55:153
27406992117,3cyclictest4157673-21kworker/u8:3+flush-259:019:20:242
27406992117,3cyclictest4157673-21kworker/u8:3+flush-259:019:20:242
27406992117,3cyclictest313367-21kworker/u8:1+flush-259:000:30:002
27404992115,4cyclictest85118-21/usr/sbin/munin20:05:221
27404992114,5cyclictest282104-21kworker/u8:4+flush-259:023:30:001
27399992117,3cyclictest48408-21kworker/u8:0+flush-259:019:30:250
27399992116,3cyclictest266623-21kworker/u8:3+flush-259:023:20:170
27399992112,8cyclictest60650irq/133-enp0s29f1:rx-000:10:140
27410992017,2cyclictest99457-21kworker/u8:3+flush-259:020:35:253
27410992017,2cyclictest360420-21kworker/u8:0+flush-259:000:35:203
27410992015,4cyclictest85143-21apt-get20:05:143
27410992014,2cyclictest349915-21kworker/u8:3+events_unbound00:20:243
27406992018,2cyclictest313367-21kworker/u8:1+events_unbound00:10:172
27406992017,2cyclictest74623-21kworker/u8:2+events_unbound19:55:252
27406992017,2cyclictest360420-21kworker/u8:0+flush-259:000:35:212
27406992017,2cyclictest183883-21kworker/u8:0+flush-259:021:50:202
27406992016,3cyclictest27386-21kworker/u8:1+events_unbound19:10:302
27404992017,2cyclictest0-21swapper/123:39:591
27404992012,6cyclictest121415-21latency_hist20:40:021
27399992016,3cyclictest99457-21kworker/u8:3+flush-259:021:09:590
27399992015,3cyclictest2698871-21systemd-journal23:40:040
27399992015,3cyclictest0-21swapper/021:29:590
27399992014,4cyclictest230202-21latency_hist22:25:010
27399992013,5cyclictest270722-21ntpq23:00:210
27410991917,2cyclictest99457-21kworker/u8:3+flush-259:020:40:273
27410991916,2cyclictest4157673-21kworker/u8:3+events_unbound19:25:213
27410991915,3cyclictest99457-21kworker/u8:3+flush-259:020:45:213
27410991915,3cyclictest59116-21kworker/u8:1+flush-259:021:10:193
27410991914,4cyclictest245995-21apt-get22:40:123
27410991913,4cyclictest2698871-21systemd-journal21:19:593
27406991916,2cyclictest162291-21kworker/u8:0+events_unbound21:27:152
27406991915,3cyclictest48408-21kworker/u8:0+events_unbound19:31:192
27406991915,3cyclictest282104-21kworker/u8:4+events_unbound23:27:412
27404991916,2cyclictest0-21swapper/121:35:011
27404991914,3cyclictest2698871-21systemd-journal23:22:391
27404991914,3cyclictest152247-21kworker/u8:4+flush-259:021:40:171
27399991916,2cyclictest75-21kswapd021:33:000
27399991914,4cyclictest60650irq/133-enp0s29f1:rx-023:35:160
27410991814,3cyclictest318749-21kworker/u8:3+events_unbound23:52:293
27410991814,3cyclictest318749-21kworker/u8:3+events_unbound23:52:283
27410991814,3cyclictest199156-21kworker/u8:1+events_unbound22:10:233
27406991816,1cyclictest38321-21kworker/u8:4+events_unbound19:45:232
27406991815,2cyclictest4157673-21kworker/u8:3+events_unbound19:28:172
27406991815,2cyclictest230430-21kworker/u8:4+events_unbound22:30:132
27406991815,2cyclictest199156-21kworker/u8:1+flush-259:022:45:202
27406991815,2cyclictest152247-21kworker/u8:4+events_unbound21:40:222
27406991814,3cyclictest59116-21kworker/u8:1+events_unbound20:33:262
27406991812,3cyclictest59116-21kworker/u8:1+events_unbound21:45:242
27404991815,1cyclictest158687-21kthreadcore21:15:191
27404991814,3cyclictest99457-21kworker/u8:3+flush-259:021:35:181
27404991814,3cyclictest99457-21kworker/u8:3+flush-259:021:35:181
27399991814,3cyclictest152247-21kworker/u8:4+flush-259:021:15:260
27399991812,2cyclictest60650irq/133-enp0s29f1:rx-022:40:270
27399991811,2cyclictest60650irq/133-enp0s29f1:rx-020:45:200
27410991714,2cyclictest48408-21kworker/u8:0+flush-259:019:30:203
27410991714,2cyclictest313367-21kworker/u8:1+events_unbound00:25:213
27410991712,2cyclictest38321-21kworker/u8:4+events_unbound19:55:143
27406991714,3cyclictest99457-21kworker/u8:3+flush-259:020:35:202
27406991714,2cyclictest48408-21kworker/u8:0+events_unbound19:51:582
27406991714,2cyclictest199156-21kworker/u8:1+events_unbound22:53:292
27406991713,3cyclictest183883-21kworker/u8:0+events_unbound22:03:282
27406991713,3cyclictest152247-21kworker/u8:4+events_unbound21:57:282
27406991710,3cyclictest59116-21kworker/u8:1+events_unbound19:40:162
27404991710,5cyclictest131686-21latency_hist20:50:001
27399991713,3cyclictest183883-21kworker/u8:0+flush-259:022:25:250
27399991713,2cyclictest193968-21turbostat21:50:000
27399991712,4cyclictest60650irq/133-enp0s29f1:rx-022:50:250
27399991711,4cyclictest2698871-21systemd-journal00:20:260
27399991711,4cyclictest245728-21latency_hist22:40:000
27399991710,2cyclictest60650irq/133-enp0s29f1:rx-023:50:260
27399991710,2cyclictest60650irq/133-enp0s29f1:rx-023:50:260
27399991710,1cyclictest60650irq/133-enp0s29f1:rx-022:02:400
27410991613,2cyclictest297660-21kworker/u8:2+events_unbound23:45:183
27406991613,2cyclictest90076-21kworker/u8:0+events_unbound20:11:182
27406991613,2cyclictest360420-21kworker/u8:0+events_unbound00:30:192
27406991613,2cyclictest27386-21kworker/u8:1+events_unbound19:15:252
27406991612,3cyclictest59116-21kworker/u8:1+events_unbound20:29:272
27406991612,3cyclictest266623-21kworker/u8:3+flush-259:023:00:172
27404991613,2cyclictest0-21swapper/123:00:001
27404991612,3cyclictest131954-21apt-get20:50:131
27404991610,4cyclictest152247-21kworker/u8:4+flush-259:021:20:181
27404991610,4cyclictest141965-21kworker/u8:2+flush-259:021:04:591
27399991614,2cyclictest60650irq/133-enp0s29f1:rx-022:17:400
27399991614,1cyclictest60650irq/133-enp0s29f1:rx-021:40:240
27399991613,2cyclictest60650irq/133-enp0s29f1:rx-022:57:150
27399991613,2cyclictest60650irq/133-enp0s29f1:rx-022:48:110
27399991613,2cyclictest60650irq/133-enp0s29f1:rx-021:00:200
27399991613,2cyclictest60650irq/133-enp0s29f1:rx-020:34:180
27399991613,2cyclictest60650irq/133-enp0s29f1:rx-020:23:360
27399991613,2cyclictest60650irq/133-enp0s29f1:rx-020:17:580
27399991613,2cyclictest60650irq/133-enp0s29f1:rx-019:20:300
27399991613,2cyclictest60650irq/133-enp0s29f1:rx-019:20:300
27406991513,2cyclictest116296-21kworker/u8:0+events_unbound21:05:182
27406991511,2cyclictest0-21swapper/223:10:002
2740499159,4cyclictest2698871-21systemd-journal00:28:021
2739999159,4cyclictest359300-21ntpq00:25:230
27399991513,1cyclictest60650irq/133-enp0s29f1:rx-023:57:260
27399991513,1cyclictest60650irq/133-enp0s29f1:rx-022:33:240
27399991513,1cyclictest60650irq/133-enp0s29f1:rx-021:10:200
27399991513,1cyclictest60650irq/133-enp0s29f1:rx-020:10:170
27399991513,1cyclictest60650irq/133-enp0s29f1:rx-019:49:550
27399991513,1cyclictest60650irq/133-enp0s29f1:rx-019:15:170
27399991513,1cyclictest60650irq/133-enp0s29f1:rx-019:10:270
27399991513,1cyclictest60650irq/133-enp0s29f1:rx-000:00:170
27399991512,2cyclictest60650irq/133-enp0s29f1:rx-019:40:170
27399991512,2cyclictest60650irq/133-enp0s29f1:rx-019:37:040
27399991512,2cyclictest60650irq/133-enp0s29f1:rx-000:19:080
27399991510,1cyclictest60650irq/133-enp0s29f1:rx-023:10:210
27406991410,3cyclictest313367-21kworker/u8:1+events_unbound23:48:292
2740499148,4cyclictest110665-21apt-get20:30:001
27404991411,2cyclictest338712-21unixbench_multi00:05:261
27404991411,2cyclictest334000-21idleruntime-cro00:05:001
27404991410,3cyclictest48420-21gpgv19:30:001
27404991410,3cyclictest230686-21fschecks_time22:25:151
27399991412,2cyclictest60650irq/133-enp0s29f1:rx-000:05:240
27399991412,1cyclictest60650irq/133-enp0s29f1:rx-023:25:180
27399991412,1cyclictest60650irq/133-enp0s29f1:rx-021:55:180
27399991412,1cyclictest60650irq/133-enp0s29f1:rx-020:05:250
27399991412,1cyclictest60650irq/133-enp0s29f1:rx-019:25:130
27399991411,2cyclictest60650irq/133-enp0s29f1:rx-023:15:530
27399991411,2cyclictest60650irq/133-enp0s29f1:rx-020:55:260
27399991410,3cyclictest60650irq/133-enp0s29f1:rx-022:10:130
2741099139,3cyclictest38321-21kworker/u8:4+flush-259:019:55:003
2740699139,3cyclictest152247-21kworker/u8:4+events_unbound21:17:572
27406991310,2cyclictest329063-21kworker/u8:4+events_unbound00:05:152
27406991310,2cyclictest240578-21kworker/u8:2+flush-259:022:55:152
27406991310,2cyclictest183883-21kworker/u8:0+flush-259:022:25:212
2740499139,3cyclictest199156-21kworker/u8:1+flush-259:021:55:261
2740499139,2cyclictest0-21swapper/123:10:171
2740499138,3cyclictest2698871-21systemd-journal20:16:591
2740499137,4cyclictest297670-21/usr/sbin/munin23:30:151
2740499136,5cyclictest2698871-21systemd-journal22:05:001
27399991311,1cyclictest60650irq/133-enp0s29f1:rx-000:31:510
2740499125,5cyclictest110665-21apt-get20:30:011
2740499121,4cyclictest328396-21head23:55:251
2739999129,2cyclictest60650irq/133-enp0s29f1:rx-021:53:020
27399991210,1cyclictest60650irq/133-enp0s29f1:rx-000:35:140
2740699117,3cyclictest90076-21kworker/u8:0+flush-259:020:15:192
2740499119,1cyclictest0-21swapper/121:05:271
2740499118,2cyclictest0-21swapper/122:05:201
2740499117,3cyclictest349915-21kworker/u8:3+flush-259:000:20:251
2740499117,2cyclictest0-21swapper/120:10:241
2740499114,5cyclictest365628-21/usr/sbin/munin00:35:151
2740499110,3cyclictest271593-21latency_hist23:05:001
2740699106,3cyclictest99457-21kworker/u8:3+events_unbound21:00:222
2740499108,1cyclictest837-21snmpd23:05:361
2740499108,1cyclictest0-21swapper/123:40:181
2740499107,1cyclictest0-21swapper/121:50:171
2740499105,4cyclictest167974-21/usr/sbin/munin21:25:141
2739999105,4cyclictest78673-21sed19:55:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional