You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-10 - 22:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Wed Dec 10, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2729407995246,2cyclictest2894847-21kworker/u8:0+events_unbound10:00:003
2729407993835,2cyclictest3010372-21kworker/u8:0+events_unbound11:40:233
2729407993835,2cyclictest2921516-21kworker/u8:0+events_unbound10:50:023
2729407993835,2cyclictest2739605-21kworker/u8:2+events_unbound07:30:033
2729407993734,2cyclictest2760179-21kworker/u8:3+events_unbound08:25:153
2729404993429,3cyclictest2984190-21kworker/u8:0+flush-259:011:15:172
2729407993330,2cyclictest3035704-21kworker/u8:3+events_unbound12:05:233
2729407993330,2cyclictest3010372-21kworker/u8:0+events_unbound12:00:153
2729404993329,3cyclictest3056146-21kworker/u8:0+flush-259:012:25:002
2729404993228,3cyclictest2718948-21kworker/u8:0+flush-259:008:25:002
2729407993128,2cyclictest2874292-21kworker/u8:5+events_unbound10:00:163
2729404993126,4cyclictest2718948-21kworker/u8:0+flush-259:007:25:002
2729404993126,3cyclictest2698871-21systemd-journal12:32:182
2729404993125,4cyclictest2760179-21kworker/u8:3+flush-259:008:00:162
2729407993027,2cyclictest3056146-21kworker/u8:0+events_unbound12:27:203
2729407992926,2cyclictest2718948-21kworker/u8:0+events_unbound08:55:013
2729407992925,3cyclictest2973141-21kworker/u8:2+events_unbound11:49:363
2729399992921,6cyclictest2698871-21systemd-journal10:25:001
2729407992825,2cyclictest3056146-21kworker/u8:0+events_unbound12:33:203
2729399992823,3cyclictest2973141-21kworker/u8:2+events_unbound11:45:181
2729399992821,5cyclictest2916154-21latency_hist10:10:001
2729397992822,4cyclictest2817149-21latency_hist08:35:010
2729397992822,4cyclictest2817149-21latency_hist08:35:010
2729397992822,4cyclictest2718948-21kworker/u8:0+flush-259:008:20:000
2729407992725,2cyclictest2947671-21kworker/u8:4+events_unbound12:23:183
2729407992724,2cyclictest2739605-21kworker/u8:2+events_unbound07:39:463
2729404992722,3cyclictest3040637-21latency_hist12:10:012
2729399992721,4cyclictest2952627-21latency_hist10:45:001
2729407992623,2cyclictest2900851-21kworker/u8:1+events_unbound10:05:283
2729404992622,3cyclictest2874292-21kworker/u8:5+flush-259:009:45:252
2729399992622,3cyclictest2864175-21kworker/u8:1+flush-259:009:40:011
2729399992619,5cyclictest2698871-21systemd-journal08:10:001
2729407992520,4cyclictest2983563-21gpgv11:15:003
2729399992522,2cyclictest2910996-21awk10:05:001
2729399992519,4cyclictest3046036-21/usr/sbin/munin12:15:151
2729397992520,4cyclictest2718948-21kworker/u8:0+flush-259:008:35:170
2729397992517,6cyclictest2698871-21systemd-journal07:55:000
2729407992422,2cyclictest2573668-21kworker/u8:3+events_unbound07:10:263
2729407992419,2cyclictest2827475-21kworker/u8:2+events_unbound08:45:023
2729404992419,3cyclictest2786483-21kworker/u8:4+flush-259:008:15:182
2729399992420,3cyclictest2760179-21kworker/u8:3+events_unbound09:25:191
2729407992320,2cyclictest2921516-21kworker/u8:0+events_unbound10:30:263
2729407992320,2cyclictest2786483-21kworker/u8:4+events_unbound08:20:023
2729407992319,3cyclictest2817393-21/usr/sbin/munin08:35:253
2729399992320,2cyclictest2947671-21kworker/u8:4+flush-259:010:50:171
2729399992319,3cyclictest2573668-21kworker/u8:3+flush-259:007:10:151
2729399992318,3cyclictest2983177-40mandb11:10:561
2729397992320,2cyclictest2832587-21awk08:49:590
2729397992317,4cyclictest2698871-21systemd-journal11:45:130
2729407992220,2cyclictest2984190-21kworker/u8:0+events_unbound11:24:563
2729407992219,2cyclictest2998472-21kworker/u8:1+events_unbound11:30:033
2729407992219,2cyclictest2718948-21kworker/u8:0+events_unbound07:48:543
2729407992219,2cyclictest2718948-21kworker/u8:0+events_unbound07:21:483
2729407992217,4cyclictest2786483-21kworker/u8:4+flush-259:008:15:003
2729404992218,3cyclictest2842749-21kworker/u8:4+flush-259:009:25:012
2729399992218,3cyclictest2947671-21kworker/u8:4+events_unbound12:10:221
2729399992216,4cyclictest2858779-21latency_hist09:15:011
2729397992216,4cyclictest2853649-21latency_hist09:10:010
2729407992118,2cyclictest2947671-21kworker/u8:4+events_unbound11:53:153
2729407992118,2cyclictest2947671-21kworker/u8:4+events_unbound11:15:033
2729407992118,2cyclictest2906036-21kworker/u8:2+events_unbound10:19:203
2729407992118,2cyclictest2771067-21kworker/u8:4+events_unbound07:52:023
2729399992118,2cyclictest2900851-21kworker/u8:1+events_unbound09:55:201
2729399992117,3cyclictest2760179-21kworker/u8:3+events_unbound09:15:241
2729399992116,3cyclictest2698871-21systemd-journal08:15:011
272940799209,3cyclictest50-21ksoftirqd/310:25:003
2729407992018,2cyclictest2760179-21kworker/u8:3+events_unbound08:30:033
2729407992018,2cyclictest2760179-21kworker/u8:3+events_unbound08:30:023
2729407992017,2cyclictest2973141-21kworker/u8:2+events_unbound11:05:273
2729407992017,2cyclictest2957776-21kworker/u8:1+events_unbound11:03:143
2729407992017,2cyclictest2760179-21kworker/u8:3+events_unbound08:52:143
2729407992017,2cyclictest2718948-21kworker/u8:0+events_unbound07:55:033
2729407992016,2cyclictest2786926-21irqcore08:05:163
2729407992013,5cyclictest2847636-21ntpq09:00:233
2729404992014,5cyclictest2968346-21apt-get11:00:112
2729404992014,4cyclictest3020036-21latency_hist11:50:012
272940499201,2cyclictest2698871-21systemd-journal11:30:002
2729399992017,2cyclictest2921516-21kworker/u8:0+flush-259:011:00:261
2729399992017,2cyclictest2760179-21kworker/u8:3+flush-259:009:20:241
2729399992015,4cyclictest2947671-21kworker/u8:4+flush-259:010:55:151
2729397992017,2cyclictest2900851-21kworker/u8:1+flush-259:010:30:250
2729397992016,3cyclictest211rcuc/008:25:000
2729397992015,4cyclictest3056146-21kworker/u8:0+flush-259:012:25:190
2729407991916,2cyclictest2786483-21kworker/u8:4+events_unbound08:17:023
2729407991916,2cyclictest2718948-21kworker/u8:0+events_unbound07:40:083
2729407991915,2cyclictest3045793-21sh12:14:593
2729404991916,2cyclictest2874292-21kworker/u8:5+flush-259:010:10:172
2729404991915,3cyclictest2921516-21kworker/u8:0+flush-259:011:05:162
2729399991916,2cyclictest2827475-21kworker/u8:2+flush-259:009:05:201
2729399991915,3cyclictest2780152-21kworker/u8:1+events_unbound08:03:541
2729399991915,2cyclictest0-21swapper/111:10:001
2729399991914,4cyclictest2973141-21kworker/u8:2+events_unbound11:25:141
2729399991913,4cyclictest2879479-21latency_hist09:35:001
2729397991915,3cyclictest2765564-21apt-get07:45:140
2729397991915,2cyclictest0-21swapper/010:15:000
2729397991914,3cyclictest211rcuc/012:40:000
2729397991913,4cyclictest2956840-21ntpq10:45:230
272939799190,17cyclictest3000063-21rs:main0
2729407991816,2cyclictest2973141-21kworker/u8:2+events_unbound11:28:143
2729407991815,2cyclictest3035704-21kworker/u8:3+events_unbound12:39:483
2729407991815,2cyclictest3035704-21kworker/u8:3+events_unbound12:15:563
2729407991815,2cyclictest0-21swapper/308:05:003
2729407991814,3cyclictest3025169-21kworker/u8:1+events_unbound11:57:133
2729407991812,4cyclictest3008819-21ntpq11:35:213
2729404991816,1cyclictest0-21swapper/208:15:002
2729404991814,3cyclictest503-21systemd-logind12:30:002
2729404991814,3cyclictest2729012-21kworker/u8:1+flush-259:007:15:272
2729404991813,4cyclictest2718948-21kworker/u8:0+flush-259:007:35:262
2729399991816,1cyclictest2771067-21kworker/u8:4+events_unbound07:55:131
2729399991815,2cyclictest2931180-21kworker/u8:3+flush-259:010:35:201
2729399991814,3cyclictest2786272-21kworker/u8:2+events_unbound08:10:011
2729399991814,3cyclictest2760179-21kworker/u8:3+flush-259:007:50:261
2729399991814,3cyclictest2718948-21kworker/u8:0+flush-259:007:35:261
2729399991813,4cyclictest2698871-21systemd-journal08:39:591
2729399991812,4cyclictest2698871-21systemd-journal11:21:131
2729399991811,5cyclictest2739594-21latency_hist07:20:001
2729397991815,2cyclictest3031020-21irqcore12:00:180
2729397991813,4cyclictest2932232-21munin-node10:25:110
272939799181,15cyclictest1-21systemd07:23:470
2729407991715,2cyclictest2874292-21kworker/u8:5+events_unbound09:47:143
2729407991714,2cyclictest2957776-21kworker/u8:1+events_unbound10:59:563
2729407991714,2cyclictest2921516-21kworker/u8:0+events_unbound10:44:413
2729407991714,2cyclictest2864175-21kworker/u8:1+events_unbound09:39:143
2729407991712,4cyclictest2942582-21apt-get10:35:123
2729404991714,2cyclictest2942326-21munin-plugin-st10:35:002
2729404991714,2cyclictest0-21swapper/208:10:002
2729404991713,3cyclictest2842749-21kworker/u8:4+flush-259:009:05:012
2729404991712,4cyclictest2931601-21gpgv10:25:002
2729399991713,3cyclictest2973141-21kworker/u8:2+events_unbound11:35:151
2729399991713,3cyclictest2894847-21kworker/u8:0+flush-259:009:50:011
2729399991713,3cyclictest2718948-21kworker/u8:0+events_unbound07:40:261
2729397991714,2cyclictest61450irq/134-enp0s29f1:tx-012:10:200
2729397991714,2cyclictest60650irq/133-enp0s29f1:rx-007:15:250
2729397991714,2cyclictest2931748-21latency_hist10:25:000
2729397991714,2cyclictest2734469-21cron07:15:010
2729397991713,3cyclictest2874958-21irqcore09:30:160
2729397991712,3cyclictest3061577-21apt-get12:30:120
272940799164,4cyclictest50-21ksoftirqd/309:30:013
2729407991614,2cyclictest2718948-21kworker/u8:0+events_unbound08:44:053
2729407991614,1cyclictest2745423-21irqcore07:25:183
2729407991613,2cyclictest2946504-21kworker/u8:2+events_unbound10:48:133
2729407991613,2cyclictest2760179-21kworker/u8:3+events_unbound09:44:573
2729407991611,3cyclictest2856197-21kthreadcore09:10:183
2729404991613,2cyclictest3000061-21in:imuxsock08:35:002
2729404991613,2cyclictest3000061-21in:imuxsock08:35:002
2729399991613,2cyclictest3035704-21kworker/u8:3+flush-259:012:05:161
2729399991613,2cyclictest2760179-21kworker/u8:3+flush-259:008:45:211
2729399991613,2cyclictest2739605-21kworker/u8:2+events_unbound07:20:241
2729399991612,3cyclictest3056146-21kworker/u8:0+flush-259:012:25:231
2729399991612,3cyclictest3056146-21kworker/u8:0+events_unbound12:30:151
2729399991610,4cyclictest2843536-21cp09:00:011
2729397991613,2cyclictest60650irq/133-enp0s29f1:rx-009:20:110
2729397991613,2cyclictest60650irq/133-enp0s29f1:rx-007:30:330
2729397991613,2cyclictest0-21swapper/011:10:000
2729397991612,3cyclictest2827475-21kworker/u8:2+flush-259:009:00:010
2729397991610,4cyclictest2905802-21latency_hist09:59:590
2729407991513,2cyclictest2931187-21kworker/u8:4+events_unbound10:26:103
2729407991512,2cyclictest2874292-21kworker/u8:5+events_unbound10:14:113
2729407991512,2cyclictest2874292-21kworker/u8:5+events_unbound09:51:563
2729407991512,2cyclictest2842749-21kworker/u8:4+events_unbound09:30:083
2729407991511,3cyclictest2760179-21kworker/u8:3+events_unbound09:06:123
2729404991513,1cyclictest0-21swapper/207:49:592
2729404991510,4cyclictest2750133-21apt-get07:30:152
272940499150,1cyclictest0-21swapper/208:50:002
272939999159,4cyclictest2698871-21systemd-journal07:26:481
2729399991512,2cyclictest3035704-21kworker/u8:3+flush-259:012:20:201
2729399991512,2cyclictest3025149-21cron11:55:001
2729399991512,2cyclictest3010372-21kworker/u8:0+events_unbound11:59:371
2729399991511,3cyclictest2946504-21kworker/u8:2+flush-259:010:45:271
272939799159,5cyclictest2811103-21ntpq08:25:200
2729397991513,1cyclictest60650irq/133-enp0s29f1:rx-011:50:250
2729397991513,1cyclictest60650irq/133-enp0s29f1:rx-011:35:160
2729397991513,1cyclictest60650irq/133-enp0s29f1:rx-011:00:140
2729397991513,1cyclictest60650irq/133-enp0s29f1:rx-010:50:170
2729397991513,1cyclictest60650irq/133-enp0s29f1:rx-010:00:500
2729397991513,1cyclictest60650irq/133-enp0s29f1:rx-009:39:460
2729397991513,1cyclictest60650irq/133-enp0s29f1:rx-009:00:210
2729397991513,1cyclictest60650irq/133-enp0s29f1:rx-007:25:120
2729397991512,2cyclictest61450irq/134-enp0s29f1:tx-011:15:250
2729397991512,2cyclictest60650irq/133-enp0s29f1:rx-012:20:100
2729397991512,2cyclictest60650irq/133-enp0s29f1:rx-010:09:260
2729397991511,2cyclictest0-21swapper/008:10:160
2729407991411,2cyclictest2729012-21kworker/u8:1+events_unbound07:18:373
2729407991411,2cyclictest0-21swapper/309:15:183
272940499147,5cyclictest2698871-21systemd-journal09:09:592
272940499145,8cyclictest3025169-21kworker/u8:1+flush-259:011:55:262
2729404991410,2cyclictest3067106-21irqcore12:35:172
2729399991410,3cyclictest3010372-21kworker/u8:0+events_unbound12:00:241
2729399991410,3cyclictest2900851-21kworker/u8:1+events_unbound10:25:141
2729399991410,3cyclictest2842749-21kworker/u8:4+events_unbound09:40:141
2729399991410,3cyclictest2816697-21unixbench_multi08:30:251
2729399991410,3cyclictest2816697-21unixbench_multi08:30:251
2729399991410,3cyclictest2760179-21kworker/u8:3+flush-259:008:25:241
2729399991410,3cyclictest2739605-21kworker/u8:2+events_unbound07:30:201
2729399991410,3cyclictest2718948-21kworker/u8:0+events_unbound07:49:591
272939799148,4cyclictest2942748-21diskstats10:35:130
2729397991413,1cyclictest60650irq/133-enp0s29f1:rx-011:55:140
2729397991413,1cyclictest60650irq/133-enp0s29f1:rx-007:40:500
2729397991412,2cyclictest60650irq/133-enp0s29f1:rx-012:18:200
2729397991412,1cyclictest60650irq/133-enp0s29f1:rx-011:45:000
2729397991412,1cyclictest60650irq/133-enp0s29f1:rx-010:55:160
2729397991412,1cyclictest60650irq/133-enp0s29f1:rx-010:40:210
2729397991412,1cyclictest60650irq/133-enp0s29f1:rx-008:42:270
2729397991412,1cyclictest60650irq/133-enp0s29f1:rx-007:35:180
2729397991411,2cyclictest61450irq/134-enp0s29f1:tx-009:45:200
2729397991411,2cyclictest60650irq/133-enp0s29f1:rx-011:26:030
2729397991411,2cyclictest2947671-21kworker/u8:4+events_unbound11:10:130
2729397991411,2cyclictest2874292-21kworker/u8:5+flush-259:009:40:230
2729397991410,3cyclictest2973141-21kworker/u8:2+flush-259:011:30:270
2729397991410,3cyclictest2859221-21fschecks_count09:15:150
272940499139,3cyclictest2906177-21cpuspeed_turbos10:00:132
2729404991310,2cyclictest2760846-21sed07:40:182
2729404991310,2cyclictest0-21swapper/211:40:132
272939999139,3cyclictest2842749-21kworker/u8:4+events_unbound10:10:251
2729399991310,2cyclictest3010372-21kworker/u8:0+events_unbound11:40:171
2729399991310,2cyclictest2973141-21kworker/u8:2+flush-259:011:15:261
2729397991311,2cyclictest60650irq/133-enp0s29f1:rx-010:18:350
2729397991311,1cyclictest60650irq/133-enp0s29f1:rx-011:20:010
272940499129,2cyclictest214-21jbd2/nvme0n1p2-809:30:162
272940499129,2cyclictest0-21swapper/208:30:002
272940499129,1cyclictest0-21swapper/212:15:212
272940499129,1cyclictest0-21swapper/210:25:162
272940499128,3cyclictest2973141-21kworker/u8:2+flush-259:011:35:242
272940499128,3cyclictest2771078-21/usr/sbin/munin07:50:152
2729404991210,1cyclictest0-21swapper/207:30:012
272939999129,2cyclictest2718948-21kworker/u8:0+events_unbound08:50:161
272939999128,3cyclictest2947671-21kworker/u8:4+events_unbound11:30:131
272939999128,3cyclictest2900851-21kworker/u8:1+events_unbound10:34:141
272939999127,3cyclictest2801979-21/usr/sbin/munin08:20:151
2729399991210,1cyclictest2821841-21kworker/u8:1+events_unbound08:55:271
2729399991210,1cyclictest0-21swapper/109:55:011
2729397991211,1cyclictest60650irq/133-enp0s29f1:rx-009:50:170
2729397991211,1cyclictest60650irq/133-enp0s29f1:rx-007:55:200
272940499118,2cyclictest2869775-21irqcore09:25:152
272940499117,2cyclictest0-21swapper/211:35:012
272939999111,8cyclictest2698871-21systemd-journal08:41:061
272939799119,1cyclictest60650irq/133-enp0s29f1:rx-008:00:120
272940499107,2cyclictest0-21swapper/208:50:162
272940499106,3cyclictest3045747-21sshd12:14:412
272940499105,4cyclictest2982617-21ntpq11:10:192
272939999106,3cyclictest2921516-21kworker/u8:0+flush-259:010:15:221
27294049996,2cyclictest2776521-21fschecks_time07:55:152
27294049993,4cyclictest2698871-21systemd-journal10:51:552
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional