You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-20 - 07:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Mar 20, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
77108228666,14sleep00-21swapper/019:07:560
76984427248,16sleep30-21swapper/319:05:133
77017326849,12sleep20-21swapper/219:05:172
77110526639,21sleep10-21swapper/119:08:131
7931192520,7sleep126-21ksoftirqd/120:10:121
8856792480,3sleep234-21ksoftirqd/200:35:012
7940472470,4sleep33056-21PK-Backend20:10:213
7746692450,4sleep33056-21PK-Backend19:15:203
8688152440,4sleep30-21swapper/323:45:143
8475152350,2sleep33056-21PK-Backend22:45:113
771384993012,16cyclictest779008-21apt19:30:103
139099270,4rtkit-daemon1389-21rtkit-daemon22:15:050
139099270,4rtkit-daemon1389-21rtkit-daemon20:04:310
139099270,4rtkit-daemon1389-21rtkit-daemon20:04:300
77137499260,24cyclictest785920-21taskset19:49:580
139099260,4rtkit-daemon1389-21rtkit-daemon22:51:080
139099260,4rtkit-daemon1389-21rtkit-daemon21:58:431
139099260,4rtkit-daemon1389-21rtkit-daemon21:58:431
139099260,4rtkit-daemon1389-21rtkit-daemon19:42:121
77137799240,22cyclictest1389-21rtkit-daemon00:06:041
77137499240,23cyclictest1389-21rtkit-daemon00:33:400
139099240,4rtkit-daemon1389-21rtkit-daemon21:36:520
139099240,21rtkit-daemon884-21gmain22:43:200
8213262230,4sleep33056-21PK-Backend21:30:013
77138499239,8cyclictest42-21ksoftirqd/323:25:163
77137799231,21cyclictest0-21swapper/119:25:301
77137499230,22cyclictest788797-21smartctl19:55:200
77137499230,22cyclictest1389-21rtkit-daemon22:07:100
77137499230,22cyclictest1389-21rtkit-daemon20:24:540
139099230,4rtkit-daemon1389-21rtkit-daemon22:58:550
139099230,4rtkit-daemon1389-21rtkit-daemon22:58:550
139099230,4rtkit-daemon1389-21rtkit-daemon19:30:451
139099230,4rtkit-daemon1389-21rtkit-daemon00:11:041
77138499228,12cyclictest3056-21PK-Backend21:00:133
771384992220,1cyclictest3056-21PK-Backend22:36:323
771384992220,1cyclictest3056-21PK-Backend22:20:513
771384992220,1cyclictest3056-21PK-Backend19:22:053
771384992219,2cyclictest3056-21PK-Backend23:51:293
771384992219,2cyclictest3056-21PK-Backend20:19:443
771384992219,2cyclictest3056-21PK-Backend20:05:453
77138499220,20cyclictest3056-21PK-Backend22:34:433
77138499220,20cyclictest3056-21PK-Backend20:28:243
77138499220,20cyclictest3056-21PK-Backend19:58:153
771382992220,1cyclictest0-21swapper/221:56:232
771382992220,1cyclictest0-21swapper/221:56:222
771382992220,1cyclictest0-21swapper/220:48:022
771382992220,1cyclictest0-21swapper/200:38:392
771382992219,2cyclictest0-21swapper/223:30:042
771382992216,4cyclictest883777-21cron00:30:002
771382992215,5cyclictest878573-21cron00:15:012
77138299220,3cyclictest1317-21systemd20:39:172
771374992219,2cyclictest0-21swapper/020:42:100
771374992212,8cyclictest791847-21cat20:05:140
139099220,4rtkit-daemon1389-21rtkit-daemon22:56:321
139099220,4rtkit-daemon1389-21rtkit-daemon22:56:311
771384992119,1cyclictest3056-21PK-Backend22:13:483
771384992119,1cyclictest3056-21PK-Backend20:50:203
771384992119,1cyclictest3056-21PK-Backend00:17:523
771384992119,1cyclictest0-21swapper/321:58:213
771384992119,1cyclictest0-21swapper/321:58:203
771384992118,2cyclictest3056-21PK-Backend19:45:443
771384992118,2cyclictest3056-21PK-Backend00:23:463
77138499210,20cyclictest0-21swapper/323:37:023
77138499210,20cyclictest0-21swapper/300:14:473
77138499210,19cyclictest3056-21PK-Backend21:35:083
771382992119,1cyclictest0-21swapper/223:05:372
771382992119,1cyclictest0-21swapper/222:35:432
771382992119,1cyclictest0-21swapper/221:48:452
771382992119,1cyclictest0-21swapper/221:11:002
771382992119,1cyclictest0-21swapper/220:44:032
771382992119,1cyclictest0-21swapper/220:01:042
771382992119,1cyclictest0-21swapper/220:01:042
771382992119,1cyclictest0-21swapper/219:45:492
771382992119,1cyclictest0-21swapper/219:14:272
771382992113,6cyclictest794597-21apt-get20:15:002
77138299210,20cyclictest0-21swapper/223:49:242
77138299210,20cyclictest0-21swapper/223:12:202
77138299210,20cyclictest0-21swapper/222:34:412
77138299210,20cyclictest0-21swapper/221:00:132
771377992118,2cyclictest0-21swapper/121:39:411
771377992110,9cyclictest812484-21unin-run21:05:001
77137799210,20cyclictest0-21swapper/120:55:401
77137799210,19cyclictest3056-21PK-Backend23:39:391
77137799210,19cyclictest3056-21PK-Backend23:09:101
77137799210,19cyclictest3056-21PK-Backend21:47:521
771374992118,2cyclictest845260-21sshd22:38:370
771374992112,7cyclictest842184-21apt22:30:100
771374992110,9cyclictest882084-21unin-run00:25:000
77137499210,20cyclictest3056-21PK-Backend21:56:460
77137499210,20cyclictest3056-21PK-Backend21:56:460
77137499210,20cyclictest0-21swapper/020:56:160
139099210,3rtkit-daemon1389-21rtkit-daemon23:11:530
139099210,3rtkit-daemon1389-21rtkit-daemon21:21:241
139099210,3rtkit-daemon1389-21rtkit-daemon20:16:380
771384992018,1cyclictest3056-21PK-Backend22:02:323
771384992018,1cyclictest3056-21PK-Backend20:00:323
771384992018,1cyclictest3056-21PK-Backend20:00:313
771384992018,1cyclictest0-21swapper/321:49:553
771384992018,1cyclictest0-21swapper/321:15:143
771384992018,1cyclictest0-21swapper/320:32:503
771384992017,2cyclictest3056-21PK-Backend23:34:123
771384992017,2cyclictest3056-21PK-Backend22:18:153
771384992012,6cyclictest918-21rs:main3
77138499200,1cyclictest3056-21PK-Backend22:09:223
77138499200,19cyclictest3056-21PK-Backend23:21:003
77138499200,19cyclictest3056-21PK-Backend23:01:403
77138499200,19cyclictest3056-21PK-Backend22:55:593
77138499200,19cyclictest3056-21PK-Backend22:55:593
77138499200,19cyclictest3056-21PK-Backend20:58:523
77138499200,19cyclictest3056-21PK-Backend20:37:153
77138499200,19cyclictest3056-21PK-Backend19:50:513
77138499200,19cyclictest3056-21PK-Backend19:11:243
77138499200,19cyclictest0-21swapper/323:40:543
77138499200,19cyclictest0-21swapper/319:29:353
77138299208,3cyclictest141rcu_preempt20:30:002
771382992018,1cyclictest0-21swapper/222:47:052
771382992018,1cyclictest0-21swapper/222:26:372
771382992018,1cyclictest0-21swapper/222:22:592
771382992018,1cyclictest0-21swapper/221:52:572
771382992018,1cyclictest0-21swapper/220:52:002
771382992018,1cyclictest0-21swapper/220:34:162
771382992018,1cyclictest0-21swapper/219:58:472
771382992018,1cyclictest0-21swapper/219:51:102
771382992017,2cyclictest3056-21PK-Backend00:06:142
771382992013,5cyclictest810728-21perf20:59:592
77138299200,1cyclictest0-21swapper/223:58:502
77138299200,1cyclictest0-21swapper/222:13:532
77138299200,1cyclictest0-21swapper/220:05:312
77138299200,19cyclictest0-21swapper/223:28:012
77138299200,19cyclictest0-21swapper/222:53:492
77138299200,19cyclictest0-21swapper/222:17:312
77138299200,19cyclictest0-21swapper/222:00:352
77138299200,19cyclictest0-21swapper/221:19:452
77138299200,19cyclictest0-21swapper/219:31:302
77138299200,19cyclictest0-21swapper/219:26:362
771377992018,1cyclictest0-21swapper/100:32:451
771377992014,4cyclictest2156-21gnome-shell22:44:091
771377992012,6cyclictest792369-21sed20:05:201
77137799200,1cyclictest3056-21PK-Backend19:37:061
77137799200,19cyclictest3056-21PK-Backend21:54:281
77137799200,19cyclictest0-21swapper/123:22:371
77137799200,19cyclictest0-21swapper/122:15:091
77137799200,19cyclictest0-21swapper/122:13:201
77137799200,19cyclictest0-21swapper/120:51:411
77137799200,18cyclictest0-21swapper/100:03:181
77137799200,18cyclictest0-21swapper/100:03:181
771374992018,1cyclictest3056-21PK-Backend21:51:130
771374992018,1cyclictest0-21swapper/023:27:440
771374992018,1cyclictest0-21swapper/022:26:200
771374992018,1cyclictest0-21swapper/022:01:460
771374992018,1cyclictest0-21swapper/019:26:340
771374992018,1cyclictest0-21swapper/019:13:350
771374992012,6cyclictest858775-21ntp_states23:15:180
77137499200,19cyclictest1389-21rtkit-daemon19:40:570
77137499200,19cyclictest0-21swapper/023:23:520
77137499200,19cyclictest0-21swapper/023:03:110
77137499200,19cyclictest0-21swapper/020:51:310
77137499200,19cyclictest0-21swapper/000:00:030
77137499200,19cyclictest0-21swapper/000:00:020
771384991917,1cyclictest3056-21PK-Backend22:43:503
771384991917,1cyclictest3056-21PK-Backend21:30:213
771384991917,1cyclictest3056-21PK-Backend20:48:443
771384991917,1cyclictest0-21swapper/323:55:343
771384991917,1cyclictest0-21swapper/321:09:263
771384991914,4cyclictest817643-21cron21:24:563
77138499190,1cyclictest3056-21PK-Backend20:23:093
77138499190,18cyclictest3056-21PK-Backend22:51:193
77138499190,18cyclictest0-21swapper/323:16:073
77138499190,18cyclictest0-21swapper/323:11:133
771382991917,1cyclictest0-21swapper/223:18:162
771382991917,1cyclictest0-21swapper/223:00:442
771382991917,1cyclictest0-21swapper/222:55:512
771382991917,1cyclictest0-21swapper/222:55:512
771382991917,1cyclictest0-21swapper/222:07:382
771382991917,1cyclictest0-21swapper/221:29:322
771382991917,1cyclictest0-21swapper/221:05:132
771382991917,1cyclictest0-21swapper/220:21:452
771382991917,1cyclictest0-21swapper/220:16:322
771382991917,1cyclictest0-21swapper/200:20:342
771382991911,6cyclictest873660-21apt00:00:132
771382991911,6cyclictest873660-21apt00:00:132
77138299190,1cyclictest3056-21PK-Backend19:38:262
77138299190,1cyclictest0-21swapper/223:37:482
77138299190,1cyclictest0-21swapper/222:42:122
77138299190,1cyclictest0-21swapper/219:20:552
771377991917,1cyclictest0-21swapper/123:48:111
771377991917,1cyclictest0-21swapper/123:26:281
771377991917,1cyclictest0-21swapper/122:53:561
771377991917,1cyclictest0-21swapper/122:37:061
771377991917,1cyclictest0-21swapper/121:11:501
771377991917,1cyclictest0-21swapper/120:22:071
771377991917,1cyclictest0-21swapper/120:00:041
771377991917,1cyclictest0-21swapper/120:00:041
771377991917,1cyclictest0-21swapper/119:10:561
771377991914,3cyclictest775264-21cron19:20:011
771377991912,5cyclictest853151-21expr23:00:121
771377991911,6cyclictest801785-21cron20:35:001
77137799190,1cyclictest0-21swapper/122:32:411
77137799190,1cyclictest0-21swapper/122:04:281
77137799190,1cyclictest0-21swapper/119:20:511
77137799190,1cyclictest0-21swapper/100:16:231
77137799190,18cyclictest0-21swapper/122:48:151
77137799190,18cyclictest0-21swapper/122:27:261
77137799190,18cyclictest0-21swapper/122:08:551
77137799190,18cyclictest0-21swapper/119:57:471
771374991917,1cyclictest0-21swapper/022:48:580
771374991912,6cyclictest879885-21sed00:15:220
771374991912,5cyclictest883025-21ntp_kernel_err00:25:190
771374991911,6cyclictest819849-21cat21:25:120
77137499190,1cyclictest3056-21PK-Backend23:40:420
77137499190,1cyclictest0-21swapper/023:51:030
77137499190,1cyclictest0-21swapper/022:10:260
77137499190,1cyclictest0-21swapper/021:12:590
77137499190,1cyclictest0-21swapper/020:35:000
77137499190,18cyclictest0-21swapper/022:21:540
77137499190,18cyclictest0-21swapper/020:49:060
77137499190,18cyclictest0-21swapper/019:39:440
139099190,15rtkit-daemon771368-21cyclictest20:27:011
771384991816,1cyclictest3056-21PK-Backend00:35:153
771384991816,1cyclictest0-21swapper/321:44:553
771384991816,1cyclictest0-21swapper/300:00:423
771384991816,1cyclictest0-21swapper/300:00:413
771384991811,6cyclictest829055-21ntp_states21:50:183
77138499180,16cyclictest3056-21PK-Backend20:41:483
77138299189,7cyclictest823865-21ntp_states21:35:192
771382991816,1cyclictest0-21swapper/219:15:282
771382991816,1cyclictest0-21swapper/200:16:282
771382991813,3cyclictest866891-21grep23:40:112
771382991811,5cyclictest826196-21sshd21:43:372
771377991816,1cyclictest0-21swapper/122:22:131
771377991816,1cyclictest0-21swapper/119:46:191
771377991813,3cyclictest1095-21runrttasks23:30:161
771377991811,5cyclictest804717-21cat20:40:221
771377991811,5cyclictest436-20systemd-journal20:36:371
771377991810,6cyclictest436-20systemd-journal00:40:001
77137499188,9cyclictest803560-21unin-run20:40:000
771374991816,1cyclictest0-21swapper/023:49:000
771374991812,5cyclictest813608-21ls21:05:210
771374991812,4cyclictest774504-21ntp_kernel_err19:15:170
771374991810,6cyclictest780487-21apt-get19:35:000
77137499180,1cyclictest0-21swapper/023:55:090
771384991715,1cyclictest3056-21PK-Backend19:42:403
771384991715,1cyclictest0-21swapper/321:12:033
77138499170,3cyclictest855433-21python323:05:203
77138499170,15cyclictest781746-21ntp_states19:35:173
771382991713,3cyclictest978-21snmpd23:53:462
77137799179,6cyclictest795518-21fw_packets20:15:161
771377991715,1cyclictest3056-21PK-Backend23:13:021
771377991715,1cyclictest0-21swapper/123:50:151
771377991714,2cyclictest3056-21PK-Backend23:19:451
771377991712,3cyclictest820443-21proc_pri21:25:201
771377991710,5cyclictest882461-21expr00:25:121
77137499179,6cyclictest822128-21ntp_states21:30:190
771374991715,1cyclictest3056-21PK-Backend21:21:310
771374991715,1cyclictest3056-21PK-Backend21:15:220
771374991715,1cyclictest3056-21PK-Backend00:06:580
771374991715,1cyclictest0-21swapper/019:20:040
771374991710,6cyclictest793811-21netstat20:10:180
771374991710,5cyclictest866428-21cron23:40:000
139099170,14rtkit-daemon0-21swapper/020:25:270
771384991614,1cyclictest3056-21PK-Backend22:26:053
771384991611,3cyclictest882298-21/usr/sbin/munin00:25:213
771384991610,5cyclictest884625-21cat00:30:183
77138299163,11cyclictest819395-21munin-plugin-st21:25:002
771382991614,1cyclictest0-21swapper/223:24:092
771382991612,3cyclictest978-21snmpd19:43:122
771382991611,3cyclictest821574-21tr21:30:142
77137799169,5cyclictest816959-21which21:15:181
771377991614,1cyclictest3056-21PK-Backend23:43:311
771377991614,1cyclictest0-21swapper/123:55:211
771377991614,1cyclictest0-21swapper/120:45:341
771377991612,3cyclictest786396-21cstates19:50:121
771377991611,3cyclictest821999-21missed_timers21:30:171
771377991610,5cyclictest812805-21ls21:05:121
771374991614,1cyclictest0-21swapper/023:05:130
771374991612,3cyclictest864082-21processes23:30:200
77137499159,4cyclictest786742-21latency19:50:150
77137499158,6cyclictest811311-21expr21:00:100
771374991511,3cyclictest877097-21acpi00:10:090
77137499151,13cyclictest828041-21cron21:49:590
771374991510,3cyclictest825049-21cstates21:40:120
771374991510,3cyclictest1095-21runrttasks00:36:590
77137799149,4cyclictest881339-21open_files00:20:181
771377991410,3cyclictest824831-21acpi21:40:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional