You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 16:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Wed Apr 29, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
274479022650,9sleep10-21swapper/107:05:221
274415522290,9sleep00-21swapper/007:05:120
28284922980,4sleep02828487-21cron11:05:000
139099940,52rtkit-daemon0-21swapper/207:09:222
274526826141,14sleep30-21swapper/307:05:493
28525682490,3sleep32852116-40unattended-upgr12:10:163
27991612470,4sleep20-21swapper/209:40:142
27933202320,9sleep013-21ksoftirqd/009:25:000
2745740992924,4cyclictest2775985-21perf08:35:003
2745733992917,10cyclictest2833970-21apt11:20:100
2745735992612,10cyclictest2852116-21unattended-upgr12:10:101
27576412250,4sleep10-21swapper/107:40:181
2745740992415,7cyclictest2787723-21users09:05:223
274574099240,11cyclictest28061712cstates10:00:123
274573899244,18cyclictest2838908-21munin-run11:35:002
2745735992415,6cyclictest436-20systemd-journal09:36:061
2745735992411,11cyclictest2793324-21turbostat.cron09:25:001
274574099232,17cyclictest2770173-21python308:15:193
2745740992311,10cyclictest2828532-21unin-run11:05:003
274574099230,21cyclictest2819826-21cron10:40:003
2745738992315,6cyclictest2818735-21df10:35:112
2745738992315,6cyclictest2771910-21sensors_temp08:20:192
2745738992314,7cyclictest2852513-21df12:10:122
2745733992321,1cyclictest0-21swapper/007:31:530
2745733992311,5cyclictest13-21ksoftirqd/008:05:500
274573399230,21cyclictest0-21swapper/010:09:420
274574099228,12cyclictest42-21ksoftirqd/308:09:573
2745738992220,1cyclictest0-21swapper/211:05:532
2745738992220,1cyclictest0-21swapper/209:16:512
274573899220,20cyclictest2784678-21systemctl09:00:002
274573899220,20cyclictest0-21swapper/211:25:342
2745735992220,1cyclictest3056-21PK-Backend07:31:401
2745735992220,1cyclictest0-21swapper/111:50:151
2745735992220,1cyclictest0-21swapper/111:48:261
2745735992220,1cyclictest0-21swapper/109:33:541
274573599220,20cyclictest2764466-21df08:00:121
274573399221,19cyclictest2811163-21munin-run10:15:000
2745733992211,4cyclictest13-21ksoftirqd/011:12:200
2745733992211,4cyclictest13-21ksoftirqd/009:55:250
2745733992210,4cyclictest13-21ksoftirqd/011:51:400
2745733992210,4cyclictest13-21ksoftirqd/008:37:460
2745733992210,4cyclictest13-21ksoftirqd/008:16:030
274574099213,2cyclictest141rcu_preempt11:29:593
274574099210,9cyclictest28503742wc12:05:163
2745738992119,1cyclictest0-21swapper/212:32:252
2745738992119,1cyclictest0-21swapper/210:46:402
2745738992119,1cyclictest0-21swapper/210:27:282
2745738992119,1cyclictest0-21swapper/210:15:442
2745738992119,1cyclictest0-21swapper/209:37:332
2745738992119,1cyclictest0-21swapper/209:31:312
2745738992119,1cyclictest0-21swapper/209:22:512
2745738992119,1cyclictest0-21swapper/209:13:262
2745738992119,1cyclictest0-21swapper/208:37:442
2745738992119,1cyclictest0-21swapper/208:01:482
2745738992119,1cyclictest0-21swapper/207:54:242
2745738992119,1cyclictest0-21swapper/207:22:542
2745738992119,1cyclictest0-21swapper/207:18:212
2745738992114,5cyclictest2851555-21mktemp12:10:002
2745738992114,5cyclictest2847561-21apt-get12:00:022
2745738992112,7cyclictest2801980-21apt-get09:50:002
274573899210,20cyclictest0-21swapper/210:07:542
274573899210,20cyclictest0-21swapper/207:12:332
274573899210,19cyclictest3056-21PK-Backend07:27:132
2745735992119,1cyclictest3056-21PK-Backend10:37:361
2745735992119,1cyclictest3056-21PK-Backend09:29:421
2745735992119,1cyclictest3056-21PK-Backend09:15:091
2745735992119,1cyclictest0-21swapper/112:29:161
2745735992119,1cyclictest0-21swapper/111:26:301
2745735992119,1cyclictest0-21swapper/107:53:031
2745735992119,1cyclictest0-21swapper/107:46:411
2745735992119,1cyclictest0-21swapper/107:14:091
2745735992118,2cyclictest3056-21PK-Backend12:18:061
2745735992118,2cyclictest0-21swapper/111:20:001
2745735992114,5cyclictest2810581-21awk10:10:221
2745735992113,6cyclictest2762766-21fw_packets07:55:161
274573599210,19cyclictest3056-21PK-Backend10:41:011
274573599210,19cyclictest3056-21PK-Backend08:09:451
274573399219,5cyclictest13-21ksoftirqd/009:06:210
274573399219,4cyclictest13-21ksoftirqd/012:18:000
274573399219,4cyclictest13-21ksoftirqd/009:26:300
274573399219,4cyclictest13-21ksoftirqd/009:11:170
2745733992119,1cyclictest3056-21PK-Backend12:36:320
2745733992119,1cyclictest3056-21PK-Backend08:30:340
2745733992119,1cyclictest3056-21PK-Backend08:21:550
2745733992119,1cyclictest0-21swapper/012:29:420
2745733992119,1cyclictest0-21swapper/011:47:580
2745733992119,1cyclictest0-21swapper/011:25:540
2745733992119,1cyclictest0-21swapper/010:28:150
2745733992119,1cyclictest0-21swapper/010:18:220
2745733992119,1cyclictest0-21swapper/009:50:510
2745733992119,1cyclictest0-21swapper/009:17:170
2745733992119,1cyclictest0-21swapper/008:54:400
2745733992119,1cyclictest0-21swapper/007:26:460
2745733992119,1cyclictest0-21swapper/007:23:480
2745733992119,1cyclictest0-21swapper/007:10:300
2745733992110,4cyclictest13-21ksoftirqd/012:24:040
2745733992110,4cyclictest13-21ksoftirqd/010:47:500
2745733992110,4cyclictest13-21ksoftirqd/010:39:200
2745733992110,4cyclictest13-21ksoftirqd/009:42:410
2745733992110,4cyclictest13-21ksoftirqd/007:17:130
274573399210,20cyclictest0-21swapper/012:14:150
274573399210,20cyclictest0-21swapper/012:00:420
274573399210,20cyclictest0-21swapper/011:40:420
274573399210,20cyclictest0-21swapper/011:05:270
274573399210,20cyclictest0-21swapper/009:49:020
274573399210,20cyclictest0-21swapper/009:34:210
274573399210,20cyclictest0-21swapper/008:28:450
274573399210,20cyclictest0-21swapper/007:56:540
139099210,14rtkit-daemon0-21swapper/207:55:292
28022662200,9sleep342-21ksoftirqd/309:50:013
274574099208,3cyclictest27678552expr08:10:133
2745740992016,3cyclictest2789836-21cron09:14:593
2745740992012,7cyclictest2807713-21apt10:05:123
2745740992012,6cyclictest2773532-21ntp_states08:25:203
2745740992012,6cyclictest2755479-21df07:35:113
274574099200,1cyclictest3056-21PK-Backend11:50:013
274574099200,18cyclictest2848100-21apt12:00:103
2745738992018,1cyclictest0-21swapper/212:35:022
2745738992018,1cyclictest0-21swapper/212:29:022
2745738992018,1cyclictest0-21swapper/211:35:542
2745738992018,1cyclictest0-21swapper/211:12:222
2745738992018,1cyclictest0-21swapper/210:53:582
2745738992018,1cyclictest0-21swapper/210:32:552
2745738992018,1cyclictest0-21swapper/208:54:122
2745738992018,1cyclictest0-21swapper/208:41:222
2745738992018,1cyclictest0-21swapper/208:07:362
2745738992018,1cyclictest0-21swapper/207:40:042
2745738992018,1cyclictest0-21swapper/207:37:402
2745738992017,2cyclictest3056-21PK-Backend11:15:142
2745738992017,2cyclictest3056-21PK-Backend09:53:132
2745738992016,3cyclictest0-21swapper/208:15:132
274573899200,19cyclictest0-21swapper/211:46:032
274573899200,19cyclictest0-21swapper/210:22:002
274573899200,19cyclictest0-21swapper/209:25:282
274573899200,19cyclictest0-21swapper/208:25:262
2745735992018,1cyclictest3056-21PK-Backend09:02:381
2745735992018,1cyclictest0-21swapper/110:00:391
2745735992017,2cyclictest3056-21PK-Backend11:43:121
2745735992015,4cyclictest2783323-21tr08:55:131
2745735992013,6cyclictest2778834-21rt-features08:40:201
2745735992013,5cyclictest2752510-21python307:25:231
274573599200,1cyclictest0-21swapper/111:00:551
274573599200,1cyclictest0-21swapper/108:54:411
274573599200,19cyclictest3056-21PK-Backend10:50:121
274573599200,19cyclictest3056-21PK-Backend10:25:111
274573599200,19cyclictest3056-21PK-Backend09:52:261
274573599200,19cyclictest0-21swapper/112:20:301
274573599200,19cyclictest0-21swapper/111:57:051
274573399209,4cyclictest13-21ksoftirqd/011:38:180
274573399209,4cyclictest13-21ksoftirqd/008:14:070
2745733992018,1cyclictest3056-21PK-Backend12:06:100
2745733992018,1cyclictest3056-21PK-Backend09:36:570
2745733992018,1cyclictest0-21swapper/012:33:060
2745733992018,1cyclictest0-21swapper/010:56:200
2745733992018,1cyclictest0-21swapper/007:41:470
2745733992013,6cyclictest2759795-21sed07:45:200
274573399200,1cyclictest3056-21PK-Backend10:20:110
274573399200,19cyclictest0-21swapper/011:57:030
274573399200,19cyclictest0-21swapper/009:00:280
274573399200,18cyclictest3056-21PK-Backend10:30:040
139099200,13rtkit-daemon3056-21PK-Backend07:14:093
274574099194,13cyclictest2749327-21seq07:16:203
274574099191,6cyclictest2841976-21awk11:40:223
2745740991914,3cyclictest2817504-21python310:30:203
2745740991912,5cyclictest436-20systemd-journal10:18:053
274574099191,16cyclictest2854747-21python312:15:213
274574099191,16cyclictest2799647-21python309:40:203
274574099190,1cyclictest3056-21PK-Backend10:54:453
274574099190,17cyclictest2820082-21/usr/sbin/munin10:40:143
2745738991917,1cyclictest0-21swapper/212:24:492
2745738991917,1cyclictest0-21swapper/211:40:342
2745738991917,1cyclictest0-21swapper/211:02:162
2745738991917,1cyclictest0-21swapper/210:13:072
2745738991917,1cyclictest0-21swapper/208:33:182
2745738991915,3cyclictest978-21snmpd11:59:522
2745738991911,6cyclictest2787399-21ntpq09:05:192
2745738991911,6cyclictest2786367-21cron09:05:002
274573899190,1cyclictest0-21swapper/211:24:332
274573899190,1cyclictest0-21swapper/210:59:252
274573899190,18cyclictest0-21swapper/211:53:192
2745735991917,1cyclictest0-21swapper/112:34:161
2745735991917,1cyclictest0-21swapper/111:31:571
2745735991917,1cyclictest0-21swapper/111:23:381
2745735991917,1cyclictest0-21swapper/110:45:121
2745735991917,1cyclictest0-21swapper/107:35:121
2745735991916,2cyclictest3056-21PK-Backend11:11:571
2745735991911,6cyclictest2851504-21date12:10:011
2745735991910,7cyclictest2781215-21unin-run08:50:011
274573599190,1cyclictest3056-21PK-Backend10:17:341
274573599190,1cyclictest0-21swapper/110:20:591
274573599190,1cyclictest0-21swapper/109:10:231
274573599190,17cyclictest3056-21PK-Backend11:35:361
274573399198,4cyclictest13-21ksoftirqd/010:42:480
2745733991917,1cyclictest0-21swapper/011:16:350
2745733991911,7cyclictest2838616-21anacron11:31:340
2745733991911,6cyclictest2780791-21awk08:45:210
274573399190,1cyclictest0-21swapper/008:40:130
2745740991815,2cyclictest3056-21PK-Backend09:24:083
2745740991813,2cyclictest42-21ksoftirqd/312:25:133
2745740991812,5cyclictest2844843-21awk11:50:163
2745740991812,5cyclictest2814096-21sed10:20:213
2745740991812,5cyclictest2795848-21cat09:30:173
2745740991811,5cyclictest42-21ksoftirqd/309:05:003
274574099181,15cyclictest2805111-21python309:55:233
2745740991811,5cyclictest2802393-21dpkg09:50:023
274574099181,15cyclictest2790854-21ntp_states09:15:183
2745740991810,6cyclictest883-21dbus-daemon07:35:003
274574099180,6cyclictest2815993-21vmstat10:25:223
274574099180,17cyclictest3056-21PK-Backend10:56:203
274574099180,16cyclictest2784636-21cron09:00:013
2745738991816,1cyclictest0-21swapper/210:00:172
2745738991816,1cyclictest0-21swapper/209:55:032
2745738991816,1cyclictest0-21swapper/208:11:002
2745735991816,1cyclictest0-21swapper/109:55:031
2745735991811,5cyclictest2768154-21load08:10:161
274573599180,17cyclictest0-21swapper/111:05:071
2745733991816,1cyclictest0-21swapper/010:01:180
2745733991811,5cyclictest2824577-21sed10:50:240
274574099179,6cyclictest42-21ksoftirqd/307:24:593
2745740991713,3cyclictest42-21ksoftirqd/312:25:003
2745740991712,3cyclictest2777102-21rt-features08:35:203
2745740991711,5cyclictest42-21ksoftirqd/309:30:003
2745740991711,5cyclictest2831106-21cat11:10:173
2745740991711,5cyclictest2828736-21sshd11:05:053
2745740991711,4cyclictest886-21NetworkManager09:40:003
2745740991710,5cyclictest2761552-21grep07:50:193
274574099170,5cyclictest2839170-21/usr/sbin/munin11:35:113
274574099170,16cyclictest2822539-21/usr/sbin/munin10:45:183
274574099170,15cyclictest0-21swapper/308:20:093
274573899179,6cyclictest2760406-21systemd07:50:012
274573899173,13cyclictest2779861-21grep08:45:122
2745738991715,1cyclictest3056-21PK-Backend07:30:582
2745738991715,1cyclictest0-21swapper/212:15:222
2745735991715,1cyclictest0-21swapper/108:38:381
2745735991714,2cyclictest3056-21PK-Backend09:42:061
2745735991712,4cyclictest2817508-21python310:30:221
2745735991710,5cyclictest436-20systemd-journal08:21:051
2745735991710,5cyclictest2860800-21sshd12:35:061
2745735991710,5cyclictest2750786-21expr07:20:201
2745733991715,1cyclictest0-21swapper/007:54:300
2745733991712,4cyclictest13-21ksoftirqd/008:05:010
274574099169,5cyclictest2782347-21sed08:50:213
274574099162,6cyclictest42-21ksoftirqd/307:55:143
2745738991614,1cyclictest0-21swapper/210:41:122
2745735991614,1cyclictest3056-21PK-Backend08:16:081
2745735991612,3cyclictest1-21systemd07:15:021
2745735991611,4cyclictest2825466-21tr10:55:131
274573599160,1cyclictest3056-21PK-Backend12:00:301
274573599160,1cyclictest3056-21PK-Backend09:08:461
274573399161,12cyclictest2756476-21file07:38:350
274573399161,11cyclictest2783188-21apt08:55:130
274574099159,4cyclictest2861351-21grep12:35:163
274574099156,7cyclictest0-21swapper/312:00:003
274574099155,8cyclictest42-21ksoftirqd/307:45:143
274574099153,2cyclictest141rcu_preempt11:23:233
2745740991510,4cyclictest3056-21PK-Backend12:34:173
2745740991510,3cyclictest27803932sleep308:45:173
274574099150,14cyclictest3056-21PK-Backend10:14:033
274574099150,13cyclictest3056-21PK-Backend07:40:053
2745735991510,4cyclictest3056-21PK-Backend09:49:111
274574099149,3cyclictest42-21ksoftirqd/308:00:053
274574099141,5cyclictest3056-21PK-Backend08:40:123
2745740991412,1cyclictest3056-21PK-Backend11:33:113
274574099140,1cyclictest3056-21PK-Backend07:26:323
274574099140,12cyclictest2833245-21grep11:15:233
274573599141,12cyclictest2775983-21cron08:34:591
2745735991410,3cyclictest3056-21PK-Backend08:26:391
274573599140,12cyclictest2809206-20unin-run10:10:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional