You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-03 - 07:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Sep 03, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
298912821250,4sleep33056-21PK-Backend20:45:123
297654721110,4sleep10-21swapper/120:10:011
29537372860,8sleep33056-21PK-Backend19:05:143
295496027840,14sleep00-21swapper/019:09:290
295355727745,22sleep10-21swapper/119:05:131
295476127638,13sleep20-21swapper/219:06:552
29951032750,4sleep13056-21PK-Backend21:00:221
29578962590,2sleep20-21swapper/219:15:112
30461832540,4sleep23056-21PK-Backend23:25:182
29894032450,7sleep013-21ksoftirqd/020:45:130
139099400,3rtkit-daemon0-21swapper/323:20:163
139099400,3rtkit-daemon0-21swapper/300:34:093
30243502360,3sleep2331rcuc/222:25:122
139099330,2rtkit-daemon0-21swapper/121:13:381
2955168992715,10cyclictest3011861-21unin-run21:50:013
295515899250,23cyclictest2955591-21expr19:10:120
2955163992419,3cyclictest3040145-21systemd23:10:002
295515899248,4cyclictest13-21ksoftirqd/019:58:500
295515899246,5cyclictest13-21ksoftirqd/019:22:290
2955158992417,5cyclictest3060938-21awk00:10:000
30086232230,4sleep13056-21PK-Backend21:40:111
2955163992319,3cyclictest34-21ksoftirqd/223:40:002
2955163992318,3cyclictest3035829-21grep22:55:172
295516099230,21cyclictest3001005-21pool-udisksd21:18:031
295515899235,4cyclictest13-21ksoftirqd/021:11:530
29894332220,4sleep2331rcuc/220:45:152
29669792220,2chrt0-21swapper/319:40:193
295516899225,15cyclictest3030382-21netstat22:40:203
2955168992211,9cyclictest42-21ksoftirqd/322:50:213
2955163992214,6cyclictest883-21dbus-daemon20:55:012
295516099226,3cyclictest141rcu_preempt23:00:021
295515899228,5cyclictest13-21ksoftirqd/021:23:140
295515899227,5cyclictest13-21ksoftirqd/000:31:250
295515899226,4cyclictest13-21ksoftirqd/022:41:350
295515899226,4cyclictest13-21ksoftirqd/020:06:550
295515899226,4cyclictest13-21ksoftirqd/020:03:390
295515899225,4cyclictest13-21ksoftirqd/021:59:230
295515899225,4cyclictest13-21ksoftirqd/021:32:540
295515899225,4cyclictest13-21ksoftirqd/019:47:230
295515899224,7cyclictest13-21ksoftirqd/020:28:410
295515899224,4cyclictest13-21ksoftirqd/023:58:020
295515899224,4cyclictest13-21ksoftirqd/022:09:070
2955158992220,1cyclictest0-21swapper/023:22:120
2955158992220,1cyclictest0-21swapper/022:35:560
2955158992220,1cyclictest0-21swapper/019:15:400
2955158992220,1cyclictest0-21swapper/000:10:260
2955158992220,1cyclictest0-21swapper/000:03:020
2955158992211,7cyclictest13-21ksoftirqd/020:10:000
295515899220,3cyclictest3004460-21chrt21:27:550
295515899220,21cyclictest0-21swapper/021:09:300
139099220,18rtkit-daemon3056-21PK-Backend19:15:201
29610112210,3sleep20-21swapper/219:25:112
2955168992119,1cyclictest0-21swapper/322:32:573
2955168992114,5cyclictest2976379-21cron20:10:003
2955168992113,6cyclictest883-21dbus-daemon00:20:013
2955168992112,7cyclictest2990769-21apt20:50:123
295516399217,3cyclictest141rcu_preempt00:05:002
295516399212,14cyclictest3042110-21apt23:15:102
2955163992118,2cyclictest3022499-21apt22:20:112
2955163992118,2cyclictest0-21swapper/220:32:092
2955163992117,3cyclictest34-21ksoftirqd/220:10:012
2955163992115,4cyclictest34-21ksoftirqd/220:15:012
295516399210,20cyclictest0-21swapper/220:44:522
295516399210,1cyclictest0-21swapper/219:32:202
295516399210,1cyclictest0-21swapper/200:27:402
2955160992114,5cyclictest2989305-21ls20:45:111
295516099210,20cyclictest0-21swapper/120:50:111
295516099210,19cyclictest3056-21PK-Backend22:36:251
295516099210,19cyclictest3056-21PK-Backend21:33:381
295516099210,19cyclictest3056-21PK-Backend20:00:501
295515899217,4cyclictest13-21ksoftirqd/022:31:560
295515899217,4cyclictest13-21ksoftirqd/019:54:570
295515899216,5cyclictest13-21ksoftirqd/020:34:060
295515899216,4cyclictest13-21ksoftirqd/022:23:490
295515899216,4cyclictest13-21ksoftirqd/019:41:450
295515899215,5cyclictest13-21ksoftirqd/023:03:240
295515899215,4cyclictest13-21ksoftirqd/023:07:410
295515899215,4cyclictest13-21ksoftirqd/022:01:210
295515899215,4cyclictest13-21ksoftirqd/021:50:410
295515899215,4cyclictest13-21ksoftirqd/021:44:570
295515899215,4cyclictest13-21ksoftirqd/021:04:380
295515899215,4cyclictest13-21ksoftirqd/019:36:080
295515899215,4cyclictest13-21ksoftirqd/019:27:310
295515899215,4cyclictest13-21ksoftirqd/000:20:550
295515899215,4cyclictest13-21ksoftirqd/000:17:540
295515899214,4cyclictest13-21ksoftirqd/023:48:550
295515899214,4cyclictest13-21ksoftirqd/023:42:340
295515899214,4cyclictest13-21ksoftirqd/023:17:070
295515899214,4cyclictest13-21ksoftirqd/022:15:120
295515899214,4cyclictest13-21ksoftirqd/021:38:150
295515899214,4cyclictest13-21ksoftirqd/020:53:310
295515899214,4cyclictest13-21ksoftirqd/019:31:270
295515899214,4cyclictest13-21ksoftirqd/000:36:360
2955158992119,1cyclictest0-21swapper/022:57:250
2955158992119,1cyclictest0-21swapper/021:15:590
2955158992119,1cyclictest0-21swapper/020:55:030
2955158992119,1cyclictest0-21swapper/020:36:180
2955158992119,1cyclictest0-21swapper/020:36:170
2955158992119,1cyclictest0-21swapper/020:23:130
2955158992119,1cyclictest0-21swapper/020:19:280
2955158992113,6cyclictest3047059-21perf23:30:000
295515899210,20cyclictest0-21swapper/023:36:190
295515899210,20cyclictest0-21swapper/022:49:060
295515899210,20cyclictest0-21swapper/022:12:440
139099210,3rtkit-daemon1389-21rtkit-daemon20:17:451
29873782200,8sleep126-21ksoftirqd/120:40:131
2955168992018,1cyclictest0-21swapper/323:53:283
2955168992018,1cyclictest0-21swapper/323:06:313
2955168992017,2cyclictest3056-21PK-Backend23:30:103
2955168992013,5cyclictest3036263-21sshd22:55:343
2955168992012,6cyclictest3031431-21awk22:45:123
295516899200,2cyclictest0-21swapper/319:15:123
295516899200,1cyclictest3056-21PK-Backend22:27:503
295516899200,1cyclictest0-21swapper/320:33:263
295516899200,19cyclictest3056-21PK-Backend23:26:583
295516899200,19cyclictest3056-21PK-Backend20:22:463
295516899200,19cyclictest0-21swapper/323:56:533
295516899200,19cyclictest0-21swapper/323:43:293
295516899200,19cyclictest0-21swapper/323:39:303
295516899200,19cyclictest0-21swapper/322:38:053
295516899200,19cyclictest0-21swapper/322:21:213
295516899200,19cyclictest0-21swapper/322:09:463
295516899200,19cyclictest0-21swapper/321:10:043
295516899200,19cyclictest0-21swapper/320:59:293
295516899200,19cyclictest0-21swapper/320:14:413
295516899200,19cyclictest0-21swapper/319:26:423
295516899200,18cyclictest3056-21PK-Backend00:09:173
295516399209,8cyclictest883-21dbus-daemon22:10:002
295516399205,12cyclictest3065534-21python300:20:192
295516399202,14cyclictest3015567-21runrttasks22:00:112
2955163992018,1cyclictest0-21swapper/223:51:372
2955163992018,1cyclictest0-21swapper/222:41:262
2955163992018,1cyclictest0-21swapper/221:37:592
2955163992018,1cyclictest0-21swapper/220:16:542
2955163992018,1cyclictest0-21swapper/200:10:152
2955163992015,3cyclictest2971187-21cut19:54:592
2955163992014,4cyclictest34-21ksoftirqd/219:15:002
2955163992012,6cyclictest3013823-21awk21:55:112
295516399200,1cyclictest0-21swapper/223:20:552
295516399200,19cyclictest3056-21PK-Backend19:55:252
295516399200,19cyclictest0-21swapper/221:24:552
295516399200,19cyclictest0-21swapper/221:08:592
295516399200,19cyclictest0-21swapper/220:29:462
295516399200,19cyclictest0-21swapper/220:20:402
295516399200,10cyclictest3011842-21perf21:50:002
2955160992018,1cyclictest3056-21PK-Backend21:51:221
2955160992018,1cyclictest3056-21PK-Backend00:36:421
2955160992018,1cyclictest0-21swapper/123:26:331
2955160992018,1cyclictest0-21swapper/123:24:101
2955160992018,1cyclictest0-21swapper/120:21:521
2955160992017,2cyclictest3056-21PK-Backend20:32:211
2955160992014,4cyclictest2156-21gnome-shell00:16:081
295516099200,19cyclictest3056-21PK-Backend23:52:411
295516099200,19cyclictest3056-21PK-Backend22:17:451
295516099200,19cyclictest3056-21PK-Backend22:14:131
295516099200,19cyclictest3056-21PK-Backend22:07:591
295516099200,19cyclictest0-21swapper/122:45:021
295516099200,19cyclictest0-21swapper/122:25:361
295516099200,19cyclictest0-21swapper/119:12:241
295516099200,19cyclictest0-21swapper/100:08:511
295516099200,18cyclictest3056-21PK-Backend23:14:021
295516099200,18cyclictest3056-21PK-Backend19:37:591
295515899204,4cyclictest13-21ksoftirqd/023:33:000
295515899204,4cyclictest13-21ksoftirqd/000:25:080
2955158992015,4cyclictest13-21ksoftirqd/022:55:000
295515899200,1cyclictest0-21swapper/023:13:270
2955168991917,1cyclictest3056-21PK-Backend19:10:323
2955168991917,1cyclictest0-21swapper/321:56:133
2955168991917,1cyclictest0-21swapper/321:54:043
295516899191,5cyclictest3070732-21python300:35:233
2955168991915,3cyclictest3065528-21python300:20:183
2955168991914,3cyclictest3043180-21users23:15:193
2955168991912,6cyclictest2968571-21expr19:45:203
295516899190,1cyclictest3056-21PK-Backend19:57:453
295516899190,1cyclictest0-21swapper/322:14:063
295516899190,1cyclictest0-21swapper/322:01:553
295516899190,1cyclictest0-21swapper/321:21:263
295516899190,1cyclictest0-21swapper/321:09:173
295516899190,1cyclictest0-21swapper/320:26:313
295516899190,18cyclictest0-21swapper/320:36:383
295516899190,18cyclictest0-21swapper/320:36:373
2955163991917,1cyclictest0-21swapper/223:59:132
2955163991917,1cyclictest0-21swapper/222:45:122
2955163991917,1cyclictest0-21swapper/222:34:432
2955163991917,1cyclictest0-21swapper/221:53:282
2955163991917,1cyclictest0-21swapper/221:13:182
2955163991917,1cyclictest0-21swapper/219:40:252
2955163991916,2cyclictest0-21swapper/220:35:202
2955163991916,2cyclictest0-21swapper/220:35:192
2955163991915,3cyclictest0-21swapper/223:50:012
2955163991913,4cyclictest978-21snmpd21:34:262
2955163991913,4cyclictest3037574-21grep23:00:162
2955163991911,6cyclictest436-20systemd-journal00:35:002
295516399191,14cyclictest2995095-21python321:00:212
295516399190,1cyclictest3056-21PK-Backend00:15:162
295516399190,1cyclictest0-21swapper/222:37:412
295516399190,1cyclictest0-21swapper/221:19:482
295516399190,1cyclictest0-21swapper/220:55:082
295516399190,1cyclictest0-21swapper/220:00:322
295516399190,18cyclictest0-21swapper/222:15:042
295516399190,18cyclictest0-21swapper/219:39:232
2955160991917,1cyclictest3056-21PK-Backend23:35:041
2955160991917,1cyclictest3056-21PK-Backend23:34:171
2955160991917,1cyclictest3056-21PK-Backend22:32:051
2955160991912,5cyclictest2977428-21sed20:10:151
295516099190,1cyclictest3056-21PK-Backend23:45:321
295516099190,1cyclictest3056-21PK-Backend21:36:221
295516099190,1cyclictest3056-21PK-Backend19:51:181
295516099190,1cyclictest3056-21PK-Backend19:45:501
295516099190,1cyclictest0-21swapper/122:53:081
295516099190,18cyclictest3056-21PK-Backend21:06:551
295516099190,18cyclictest3056-21PK-Backend19:44:011
295516099190,18cyclictest3056-21PK-Backend19:31:091
295516099190,18cyclictest0-21swapper/122:00:221
295516099190,18cyclictest0-21swapper/120:25:371
295516099190,18cyclictest0-21swapper/119:57:181
295515899195,4cyclictest13-21ksoftirqd/023:53:120
2955158991917,1cyclictest0-21swapper/020:40:170
2955158991911,6cyclictest13-21ksoftirqd/022:25:110
2955168991816,1cyclictest0-21swapper/323:01:103
2955168991816,1cyclictest0-21swapper/321:03:003
2955168991816,1cyclictest0-21swapper/320:02:523
2955168991813,3cyclictest2970293-21memory19:50:183
2955168991812,4cyclictest1009-21lldpd21:33:263
2955168991811,5cyclictest2964652-21tr19:35:113
2955168991810,6cyclictest436-20systemd-journal00:25:173
2955163991816,1cyclictest0-21swapper/222:51:552
2955163991816,1cyclictest0-21swapper/221:41:302
2955163991816,1cyclictest0-21swapper/200:06:022
2955163991812,5cyclictest3070725-21gzip00:35:212
2955163991811,5cyclictest34-21ksoftirqd/221:25:152
2955163991810,6cyclictest3050787-21apt23:40:122
295516399180,1cyclictest0-21swapper/222:12:062
2955160991816,1cyclictest3056-21PK-Backend21:26:281
2955160991816,1cyclictest3056-21PK-Backend20:58:101
2955160991816,1cyclictest0-21swapper/122:23:261
2955160991816,1cyclictest0-21swapper/100:20:131
2955160991811,5cyclictest3067882-21basename00:30:001
2955160991811,5cyclictest3010500-21grep21:45:121
2955160991811,4cyclictest1095-21runrttasks00:10:121
295516899179,6cyclictest3053294-21awk23:45:183
2955168991715,1cyclictest3056-21PK-Backend20:40:373
2955168991712,3cyclictest3058101-21df_inode00:00:113
2955168991710,5cyclictest2978614-21find20:15:103
295516899170,5cyclictest2963359-21sed19:30:173
295516899170,4cyclictest3008904-21ls21:40:133
295516899170,3cyclictest2960769-21munin-run19:25:013
295516899170,15cyclictest3040391-21cpu23:10:173
2955163991715,1cyclictest0-21swapper/219:48:422
2955163991712,4cyclictest978-21snmpd19:21:232
2955163991711,4cyclictest34-21ksoftirqd/223:35:002
295516099176,2cyclictest141rcu_preempt23:05:111
295516099174,2cyclictest141rcu_preempt00:00:021
2955160991715,1cyclictest0-21swapper/120:38:291
2955160991715,1cyclictest0-21swapper/120:38:291
2955160991714,2cyclictest3056-21PK-Backend22:57:401
295516099170,16cyclictest3056-21PK-Backend19:21:231
2955158991715,1cyclictest0-21swapper/021:49:590
2955168991614,1cyclictest0-21swapper/322:17:033
2955168991614,1cyclictest0-21swapper/321:35:193
295516899160,14cyclictest3003175-21apt21:25:123
295516399169,6cyclictest3041393-21wc23:10:272
295516099169,5cyclictest3059361-21ls00:05:021
2955160991614,1cyclictest3056-21PK-Backend23:17:131
2955160991613,2cyclictest3056-21PK-Backend23:42:351
2955160991613,2cyclictest3056-21PK-Backend22:42:521
2955160991611,3cyclictest2961085-21cpuspeed19:25:111
295516899154,2cyclictest141rcu_preempt21:15:183
295516899151,2cyclictest141rcu_preempt00:10:203
295516099159,4cyclictest3068320-21tr00:30:131
295516099159,4cyclictest3013885-21cpuspeed21:55:121
2955160991511,3cyclictest3001417-21/usr/sbin/munin21:20:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional