You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-12 - 04:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Jul 12, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
246888021440,3sleep33056-21PK-Backend22:00:003
243421621180,3sleep13056-21PK-Backend20:20:131
240828527940,13sleep10-21swapper/119:06:261
240712227948,21sleep00-21swapper/019:05:140
139099780,62rtkit-daemon0-21swapper/219:05:112
240823227638,14sleep30-21swapper/319:05:453
24654362520,4sleep30-21swapper/321:50:013
25102692460,4sleep13056-21PK-Backend23:55:221
24903752270,3chrt3056-21PK-Backend23:00:113
240872699262,8cyclictest2465692-21apt21:50:113
25065892250,6sleep00-21swapper/023:45:190
25091622240,3sleep00-21swapper/023:55:010
240872699241,21cyclictest2515435-21/usr/sbin/munin00:10:183
2408726992410,12cyclictest1095-21runrttasks23:50:123
24348962230,4sleep20-21swapper/220:20:212
2408726992320,2cyclictest3056-21PK-Backend22:59:343
2408726992315,6cyclictest2445789-21apt-get20:55:003
240871899230,21cyclictest3056-21PK-Backend21:53:531
139099230,16rtkit-daemon3056-21PK-Backend20:33:583
139099230,15rtkit-daemon0-21swapper/300:22:353
2408726992220,1cyclictest3056-21PK-Backend23:06:233
2408726992220,1cyclictest3056-21PK-Backend23:06:233
2408726992219,2cyclictest3056-21PK-Backend23:30:093
2408726992219,2cyclictest3056-21PK-Backend23:15:573
2408726992219,2cyclictest0-21swapper/322:30:213
2408726992214,6cyclictest2520128-21grep00:25:123
2408722992220,1cyclictest0-21swapper/223:45:122
2408722992220,1cyclictest0-21swapper/223:17:132
2408722992220,1cyclictest0-21swapper/222:16:032
2408722992220,1cyclictest0-21swapper/222:14:072
2408722992220,1cyclictest0-21swapper/221:45:352
2408722992220,1cyclictest0-21swapper/220:59:102
2408722992220,1cyclictest0-21swapper/219:40:432
2408722992220,1cyclictest0-21swapper/200:39:062
2408722992217,3cyclictest2486486-21taskset22:49:562
2408722992214,6cyclictest2468498-21users21:55:252
240872299220,3cyclictest2513421-21df00:05:122
240872299220,2cyclictest2509313-21apt23:55:132
240872299220,21cyclictest0-21swapper/221:03:372
2408718992220,1cyclictest3056-21PK-Backend20:55:181
2408718992220,1cyclictest0-21swapper/100:28:241
2408718992219,2cyclictest3056-21PK-Backend23:48:291
2408718992213,7cyclictest883-21dbus-daemon23:35:011
2408716992215,5cyclictest2156-21gnome-shell22:02:090
240871699220,20cyclictest0-21swapper/019:50:170
139099220,15rtkit-daemon3056-21PK-Backend23:14:593
139099220,15rtkit-daemon3056-21PK-Backend22:00:173
139099220,15rtkit-daemon0-21swapper/220:26:412
139099220,14rtkit-daemon0-21swapper/322:44:253
139099220,14rtkit-daemon0-21swapper/220:47:252
24925232210,3sleep1251rcuc/123:05:181
24925232210,3sleep1251rcuc/123:05:171
240872699212,4cyclictest0-21swapper/319:10:123
2408726992119,1cyclictest3056-21PK-Backend23:24:423
2408726992119,1cyclictest3056-21PK-Backend22:49:533
2408726992119,1cyclictest3056-21PK-Backend22:15:403
2408726992119,1cyclictest0-21swapper/323:58:553
2408726992119,1cyclictest0-21swapper/322:23:443
2408726992119,1cyclictest0-21swapper/320:45:023
2408726992119,1cyclictest0-21swapper/320:20:353
2408726992118,2cyclictest3056-21PK-Backend19:45:333
2408726992113,6cyclictest2499766-21python323:25:213
2408726992111,7cyclictest2028-21upowerd00:35:583
240872699210,8cyclictest2511271-21cnf-update-db00:00:013
240872699210,20cyclictest3056-21PK-Backend19:28:043
240872699210,19cyclictest3056-21PK-Backend19:55:473
240872699210,19cyclictest3056-21PK-Backend19:30:203
2408722992119,1cyclictest0-21swapper/223:53:032
2408722992119,1cyclictest0-21swapper/223:40:522
2408722992119,1cyclictest0-21swapper/223:27:352
2408722992119,1cyclictest0-21swapper/223:01:122
2408722992119,1cyclictest0-21swapper/222:31:312
2408722992119,1cyclictest0-21swapper/222:03:182
2408722992119,1cyclictest0-21swapper/221:52:302
2408722992119,1cyclictest0-21swapper/221:09:592
2408722992119,1cyclictest0-21swapper/219:47:532
2408722992119,1cyclictest0-21swapper/219:31:362
2408722992119,1cyclictest0-21swapper/200:23:392
2408722992113,6cyclictest2516664-21expr00:15:112
240872299210,20cyclictest0-21swapper/223:06:052
240872299210,20cyclictest0-21swapper/223:06:042
240872299210,20cyclictest0-21swapper/221:39:132
240872299210,20cyclictest0-21swapper/220:52:142
240872299210,20cyclictest0-21swapper/220:18:422
2408718992119,1cyclictest3056-21PK-Backend22:29:411
2408718992119,1cyclictest3056-21PK-Backend21:37:091
2408718992119,1cyclictest3056-21PK-Backend21:31:011
2408718992119,1cyclictest3056-21PK-Backend21:27:571
2408718992119,1cyclictest0-21swapper/123:41:261
2408718992119,1cyclictest0-21swapper/123:19:231
2408718992119,1cyclictest0-21swapper/122:45:581
2408718992119,1cyclictest0-21swapper/122:08:531
2408718992119,1cyclictest0-21swapper/120:33:011
2408718992114,5cyclictest436-20systemd-journal22:55:001
2408718992113,6cyclictest2420417-21df19:40:121
240871899210,20cyclictest1389-21rtkit-daemon23:13:021
240871899210,20cyclictest0-21swapper/122:21:241
240871899210,19cyclictest3056-21PK-Backend21:45:141
240871899210,19cyclictest3056-21PK-Backend21:11:281
240871899210,19cyclictest3056-21PK-Backend21:00:521
240871899210,19cyclictest3056-21PK-Backend19:12:441
240871699212,16cyclictest2426729-21idleruntime-cro20:00:000
2408716992114,5cyclictest2451231-21cron21:09:590
240871699210,20cyclictest0-21swapper/021:29:000
240871699210,20cyclictest0-21swapper/000:32:580
139099210,15rtkit-daemon0-21swapper/321:26:453
240872699207,11cyclictest42-21ksoftirqd/323:40:203
2408726992018,1cyclictest3056-21PK-Backend19:37:163
2408726992018,1cyclictest0-21swapper/323:49:433
2408726992018,1cyclictest0-21swapper/322:28:593
2408726992018,1cyclictest0-21swapper/321:43:083
2408726992018,1cyclictest0-21swapper/321:35:393
2408726992018,1cyclictest0-21swapper/321:30:393
2408726992018,1cyclictest0-21swapper/321:02:333
2408726992018,1cyclictest0-21swapper/320:19:133
2408726992018,1cyclictest0-21swapper/319:18:023
2408726992016,2cyclictest2454707-21taskset21:19:293
2408726992015,3cyclictest2429582-21python320:05:213
240872699200,1cyclictest3056-21PK-Backend23:36:103
240872699200,1cyclictest3056-21PK-Backend19:51:553
240872699200,1cyclictest3056-21PK-Backend19:41:423
240872699200,1cyclictest3056-21PK-Backend00:30:393
240872699200,19cyclictest3056-21PK-Backend22:53:463
240872699200,19cyclictest0-21swapper/322:35:343
240872699200,19cyclictest0-21swapper/320:43:063
240872699200,19cyclictest0-21swapper/320:12:373
240872699200,19cyclictest0-21swapper/300:06:463
240872699200,18cyclictest3056-21PK-Backend22:05:063
2408722992018,1cyclictest0-21swapper/222:41:252
2408722992018,1cyclictest0-21swapper/221:16:212
2408722992018,1cyclictest0-21swapper/220:30:172
2408722992018,1cyclictest0-21swapper/220:06:522
240872299200,1cyclictest0-21swapper/219:16:502
240872299200,19cyclictest0-21swapper/223:31:402
240872299200,19cyclictest0-21swapper/222:56:312
240872299200,19cyclictest0-21swapper/222:20:282
240872299200,19cyclictest0-21swapper/221:41:092
240872299200,19cyclictest0-21swapper/221:34:112
240872299200,19cyclictest0-21swapper/221:11:552
240872299200,19cyclictest0-21swapper/220:10:102
240872299200,19cyclictest0-21swapper/219:39:212
240872299200,19cyclictest0-21swapper/219:11:572
240872299200,19cyclictest0-21swapper/200:27:102
2408718992018,1cyclictest3056-21PK-Backend22:34:551
2408718992018,1cyclictest3056-21PK-Backend21:20:411
2408718992018,1cyclictest3056-21PK-Backend21:17:021
2408718992018,1cyclictest3056-21PK-Backend21:05:531
2408718992018,1cyclictest3056-21PK-Backend20:46:411
2408718992018,1cyclictest3056-21PK-Backend20:29:581
2408718992018,1cyclictest3056-21PK-Backend20:16:051
2408718992018,1cyclictest3056-21PK-Backend19:23:401
2408718992018,1cyclictest0-21swapper/122:59:291
2408718992018,1cyclictest0-21swapper/122:03:521
2408718992018,1cyclictest0-21swapper/119:59:151
2408718992018,1cyclictest0-21swapper/100:31:571
2408718992014,5cyclictest2504065-21sed23:40:021
2408718992012,6cyclictest2430177-21cron20:09:591
2408718992011,7cyclictest26-21ksoftirqd/119:50:121
240871899200,1cyclictest0-21swapper/120:35:311
240871899200,19cyclictest3056-21PK-Backend21:42:431
240871899200,19cyclictest3056-21PK-Backend20:52:491
240871899200,19cyclictest0-21swapper/119:35:281
240871899200,18cyclictest3056-21PK-Backend22:36:511
2408716992018,1cyclictest0-21swapper/023:35:340
2408716992018,1cyclictest0-21swapper/021:48:390
2408716992011,7cyclictest2431174-21awk20:10:160
240871699200,1cyclictest0-21swapper/023:30:050
240871699200,1cyclictest0-21swapper/023:26:000
240871699200,1cyclictest0-21swapper/022:41:320
240871699200,1cyclictest0-21swapper/019:35:370
240871699200,19cyclictest3056-21PK-Backend00:02:440
240871699200,19cyclictest0-21swapper/022:27:410
240871699200,19cyclictest0-21swapper/021:38:320
240871699200,19cyclictest0-21swapper/019:31:110
240871699200,19cyclictest0-21swapper/000:29:270
139099200,14rtkit-daemon0-21swapper/321:23:243
139099200,14rtkit-daemon0-21swapper/321:10:273
139099200,14rtkit-daemon0-21swapper/319:23:573
139099200,14rtkit-daemon0-21swapper/222:53:422
139099200,14rtkit-daemon0-21swapper/219:20:032
2408726991917,1cyclictest0-21swapper/321:06:593
2408726991917,1cyclictest0-21swapper/320:55:033
2408726991917,1cyclictest0-21swapper/320:03:323
240872699190,1cyclictest0-21swapper/320:27:323
240872299192,15cyclictest2471090-21cpuspeed_turbos22:05:132
2408722991917,1cyclictest0-21swapper/220:39:442
2408722991917,1cyclictest0-21swapper/220:01:382
2408722991917,1cyclictest0-21swapper/219:51:112
2408722991917,1cyclictest0-21swapper/200:34:122
2408722991917,1cyclictest0-21swapper/200:01:412
2408722991911,6cyclictest2440859-21apt20:40:122
240872299190,1cyclictest0-21swapper/223:10:582
240872299190,18cyclictest0-21swapper/221:23:032
240872299190,18cyclictest0-21swapper/200:14:252
2408718991917,1cyclictest3056-21PK-Backend23:50:391
2408718991917,1cyclictest3056-21PK-Backend00:14:321
2408718991917,1cyclictest0-21swapper/123:02:131
2408718991917,1cyclictest0-21swapper/119:45:091
2408718991913,4cyclictest2156-21gnome-shell00:02:101
2408718991911,6cyclictest2475351-21cat22:15:211
240871899190,1cyclictest0-21swapper/122:11:231
240871899190,1cyclictest0-21swapper/121:56:231
240871899190,1cyclictest0-21swapper/119:31:361
240871899190,18cyclictest2498620-21cron23:25:001
2408716991917,1cyclictest0-21swapper/023:13:430
2408716991917,1cyclictest0-21swapper/023:01:260
2408716991917,1cyclictest0-21swapper/022:51:470
2408716991917,1cyclictest0-21swapper/022:39:020
2408716991917,1cyclictest0-21swapper/022:16:310
2408716991917,1cyclictest0-21swapper/022:09:350
2408716991917,1cyclictest0-21swapper/021:52:530
2408716991917,1cyclictest0-21swapper/021:43:530
2408716991917,1cyclictest0-21swapper/021:33:120
2408716991917,1cyclictest0-21swapper/021:13:110
2408716991917,1cyclictest0-21swapper/000:35:090
2408716991917,1cyclictest0-21swapper/000:11:440
2408716991914,3cyclictest2442768-21tr20:45:120
2408716991911,6cyclictest2477114-21expr22:20:220
240871699190,1cyclictest0-21swapper/019:25:300
240871699190,18cyclictest0-21swapper/023:21:550
240871699190,18cyclictest0-21swapper/023:06:530
240871699190,18cyclictest0-21swapper/023:06:530
240871699190,18cyclictest0-21swapper/022:12:050
240871699190,18cyclictest0-21swapper/021:56:310
240871699190,18cyclictest0-21swapper/020:37:010
240871699190,18cyclictest0-21swapper/020:17:270
2408726991816,1cyclictest3056-21PK-Backend22:10:053
2408726991810,6cyclictest2516926-21which00:15:153
2408722991816,1cyclictest0-21swapper/222:38:072
2408722991816,1cyclictest0-21swapper/221:28:372
2408722991816,1cyclictest0-21swapper/219:56:242
2408722991812,5cyclictest883-21dbus-daemon22:28:032
2408718991816,1cyclictest3056-21PK-Backend22:40:101
2408718991816,1cyclictest3056-21PK-Backend00:19:251
2408718991816,1cyclictest3056-21PK-Backend00:09:401
2408718991816,1cyclictest0-21swapper/123:28:561
2408718991816,1cyclictest0-21swapper/119:25:221
2408718991811,6cyclictest2428452-21turbostat.cron20:04:591
2408718991810,6cyclictest2524182-21ntpq00:35:201
2408718991810,2cyclictest26-21ksoftirqd/100:25:001
2408716991813,4cyclictest2438806-21sshd20:34:340
2408716991813,3cyclictest2495522-21cpuspeed23:15:140
2408716991812,5cyclictest2445320-21sed20:50:230
2408716991812,4cyclictest2514523-21perf00:10:000
2408716991811,5cyclictest2420462-21ls19:40:140
2408716991810,6cyclictest2422524-21ntp_states19:45:180
240871699180,1cyclictest3056-21PK-Backend23:41:150
240871699180,1cyclictest0-21swapper/020:58:380
240871699180,17cyclictest0-21swapper/020:25:330
2408726991712,3cyclictest2440620-21munin-run20:40:013
2408722991715,1cyclictest0-21swapper/223:36:332
2408722991711,5cyclictest883-21dbus-daemon19:30:002
2408718991715,1cyclictest3056-21PK-Backend20:10:171
2408718991713,2cyclictest26-21ksoftirqd/120:40:091
240871699174,11cyclictest2489177-21needreboot22:55:190
2408716991715,1cyclictest0-21swapper/022:45:590
2408716991715,1cyclictest0-21swapper/021:01:080
2408716991715,1cyclictest0-21swapper/020:44:520
2408716991715,1cyclictest0-21swapper/019:24:560
2408716991715,1cyclictest0-21swapper/019:17:590
2408716991715,1cyclictest0-21swapper/019:14:140
2408716991711,4cyclictest2429454-21cat20:05:180
2408716991710,5cyclictest2511037-21sort00:00:000
240871699170,15cyclictest2435391-21munin-run20:25:000
2408722991614,1cyclictest0-21swapper/223:20:462
2408718991614,1cyclictest3056-21PK-Backend19:17:381
240871699169,5cyclictest2517982-21cron00:20:010
2408716991614,1cyclictest0-21swapper/022:31:320
2408716991611,3cyclictest2454778-21grep21:19:590
240871699160,15cyclictest3056-21PK-Backend20:02:280
2408716991511,3cyclictest2518238-21/usr/sbin/munin00:20:180
2408716991510,3cyclictest2156-21gnome-shell21:20:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional