You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 05:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue May 26, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
100182821230,4sleep00-21swapper/020:40:140
100182821230,4sleep00-21swapper/020:40:140
107936821200,3sleep13056-21PK-Backend00:20:121
105813021100,6sleep342-21ksoftirqd/323:20:123
96885728339,14sleep10-21swapper/119:06:401
96911927652,15sleep00-21swapper/019:10:010
96884927640,13sleep30-21swapper/319:06:343
96767526241,14sleep20-21swapper/219:05:122
9881832520,4sleep234-21ksoftirqd/220:00:182
10086272490,3sleep20-21swapper/221:00:132
10424882360,4sleep03056-21PK-Backend22:35:140
9696972340,2sleep013-21ksoftirqd/019:10:120
969276992612,12cyclictest1043562-21unin-run22:40:003
969266992615,9cyclictest977886-21ntp_states19:30:201
969276992513,10cyclictest1061399-21apt23:30:113
969276992513,10cyclictest1061399-21apt23:30:113
96927699242,6cyclictest1003159-21apt20:45:123
969272992410,7cyclictest34-21ksoftirqd/223:35:182
969262992410,12cyclictest2164-21gdbus20:37:130
139099240,16rtkit-daemon3056-21PK-Backend20:45:190
139099240,16rtkit-daemon3056-21PK-Backend19:15:380
969276992310,4cyclictest42-21ksoftirqd/320:19:193
969272992321,1cyclictest0-21swapper/219:29:182
969272992316,2cyclictest34-21ksoftirqd/222:21:402
96927299230,21cyclictest1046154-21df22:45:142
969266992316,5cyclictest26-21ksoftirqd/119:45:111
139099230,16rtkit-daemon13-21ksoftirqd/023:27:280
96927699221,2cyclictest141rcu_preempt20:45:003
96927699221,2cyclictest141rcu_preempt20:45:003
969276992210,4cyclictest42-21ksoftirqd/300:37:223
969272992220,1cyclictest0-21swapper/221:09:562
969272992214,6cyclictest1-21systemd00:05:012
969272992210,3cyclictest34-21ksoftirqd/200:11:582
96926699220,3cyclictest1084833-21cat00:35:181
96926699220,3cyclictest1054856-21hddtemp_smartct23:10:141
96926699220,20cyclictest3056-21PK-Backend20:41:031
96926699220,20cyclictest3056-21PK-Backend20:41:031
969262992215,6cyclictest1073536-21cron00:04:590
969262992212,7cyclictest1034725-21sshd22:13:290
96926299220,3cyclictest1032145-21ntp_states22:05:180
10459052220,3sleep03056-21PK-Backend22:45:130
9966982210,4sleep00-21swapper/020:25:180
9761492210,6chrt0-21swapper/019:25:190
96927699214,15cyclictest1079596-21ls00:20:183
969276992116,4cyclictest42-21ksoftirqd/321:40:003
969276992116,3cyclictest1022742-21apt21:40:113
969276992113,6cyclictest999410-21perf20:35:003
969276992113,6cyclictest995217-21ntp_states20:20:193
969276992113,6cyclictest883-21dbus-daemon23:05:013
96927699211,18cyclictest1076400-21python300:10:193
96927699210,1cyclictest3056-21PK-Backend23:12:413
96927299219,3cyclictest141rcu_preempt23:14:432
96927299218,3cyclictest34-21ksoftirqd/222:05:102
969272992119,1cyclictest3056-21PK-Backend23:23:252
969272992119,1cyclictest0-21swapper/223:45:372
969272992119,1cyclictest0-21swapper/222:57:172
969272992119,1cyclictest0-21swapper/221:47:002
969272992119,1cyclictest0-21swapper/221:42:002
969272992119,1cyclictest0-21swapper/219:57:092
969272992119,1cyclictest0-21swapper/219:52:092
969272992119,1cyclictest0-21swapper/219:16:132
969272992119,1cyclictest0-21swapper/200:33:222
969272992116,4cyclictest34-21ksoftirqd/223:07:142
969272992114,6cyclictest1019773-21awk21:30:192
969272992114,5cyclictest34-21ksoftirqd/222:00:012
969272992113,6cyclictest436-20systemd-journal21:40:002
969272992113,6cyclictest1069787-21apt-get23:55:002
96927299210,20cyclictest0-21swapper/222:29:122
96927299210,20cyclictest0-21swapper/222:14:182
96926699214,16cyclictest1052458-21munin-run23:05:001
969266992119,1cyclictest3056-21PK-Backend22:39:461
969266992119,1cyclictest3056-21PK-Backend21:34:291
969266992118,2cyclictest3056-21PK-Backend20:14:261
969266992118,2cyclictest0-21swapper/122:15:191
969266992116,3cyclictest1003347-21tr20:45:131
96926699210,19cyclictest3056-21PK-Backend22:56:151
969262992119,1cyclictest0-21swapper/021:13:270
969262992110,9cyclictest978604-21apt19:35:000
96927699208,4cyclictest42-21ksoftirqd/323:48:103
96927699203,15cyclictest1015269-21cron21:20:003
969276992018,1cyclictest0-21swapper/321:29:543
969276992018,1cyclictest0-21swapper/321:22:513
969276992015,3cyclictest1074179-21diskstats00:05:143
969276992014,3cyclictest1028596-21grep21:55:193
969276992013,5cyclictest1063109-21find23:35:093
969276992012,6cyclictest988402-21python320:00:223
969276992012,6cyclictest1026942-21ntp_states21:50:193
96927699200,1cyclictest3056-21PK-Backend19:36:393
96927699200,19cyclictest3056-21PK-Backend21:08:113
96927699200,19cyclictest3056-21PK-Backend19:12:193
96927699200,19cyclictest0-21swapper/323:43:383
96927699200,19cyclictest0-21swapper/323:16:483
96927699200,19cyclictest0-21swapper/322:48:363
96927699200,19cyclictest0-21swapper/319:30:033
96927299207,3cyclictest34-21ksoftirqd/223:30:172
96927299207,3cyclictest34-21ksoftirqd/223:30:172
96927299206,3cyclictest141rcu_preempt21:26:222
969272992018,1cyclictest0-21swapper/222:51:152
969272992018,1cyclictest0-21swapper/220:20:282
969272992018,1cyclictest0-21swapper/219:42:092
969272992018,1cyclictest0-21swapper/219:36:072
969272992018,1cyclictest0-21swapper/219:33:302
969272992012,6cyclictest1044794-21ntp_states22:40:192
96927299200,19cyclictest0-21swapper/223:59:502
96927299200,19cyclictest0-21swapper/223:25:362
96927299200,19cyclictest0-21swapper/223:19:072
96927299200,19cyclictest0-21swapper/222:31:422
96927299200,19cyclictest0-21swapper/221:24:502
96927299200,19cyclictest0-21swapper/220:50:172
96927299200,19cyclictest0-21swapper/220:39:342
96927299200,19cyclictest0-21swapper/220:31:232
96927299200,19cyclictest0-21swapper/220:16:162
96927299200,19cyclictest0-21swapper/220:11:492
96927299200,19cyclictest0-21swapper/200:16:052
96927299200,19cyclictest0-21swapper/200:08:212
96927299200,11cyclictest34-21ksoftirqd/221:13:092
969266992018,1cyclictest3056-21PK-Backend22:46:211
969266992018,1cyclictest3056-21PK-Backend22:41:081
969266992018,1cyclictest3056-21PK-Backend00:12:541
969266992018,1cyclictest0-21swapper/119:41:141
969266992017,2cyclictest3056-21PK-Backend20:16:151
96926699200,1cyclictest3056-21PK-Backend21:43:281
96926699200,1cyclictest3056-21PK-Backend19:20:251
96926699200,19cyclictest3056-21PK-Backend23:17:041
96926699200,19cyclictest3056-21PK-Backend22:08:431
96926699200,19cyclictest3056-21PK-Backend21:46:331
96926699200,19cyclictest3056-21PK-Backend20:08:241
96926699200,19cyclictest0-21swapper/121:25:371
96926699200,19cyclictest0-21swapper/121:10:091
96926699200,19cyclictest0-21swapper/120:36:361
96926699200,18cyclictest3056-21PK-Backend20:25:411
96926699200,18cyclictest3056-21PK-Backend00:30:311
969262992017,2cyclictest0-21swapper/019:36:270
96926299200,18cyclictest1038338-21basename22:25:000
969276991917,1cyclictest3056-21PK-Backend22:17:193
969276991917,1cyclictest3056-21PK-Backend19:40:033
969276991917,1cyclictest0-21swapper/322:57:213
969276991917,1cyclictest0-21swapper/322:54:373
969276991917,1cyclictest0-21swapper/322:43:563
969276991917,1cyclictest0-21swapper/322:33:423
969276991917,1cyclictest0-21swapper/321:46:583
969276991917,1cyclictest0-21swapper/321:04:053
969276991917,1cyclictest0-21swapper/320:52:023
969276991917,1cyclictest0-21swapper/320:26:343
969276991917,1cyclictest0-21swapper/320:14:313
969276991917,1cyclictest0-21swapper/320:05:043
969276991917,1cyclictest0-21swapper/319:25:303
969276991913,5cyclictest42-21ksoftirqd/322:10:013
969276991912,5cyclictest1068727-21smartctl23:50:153
96927699190,9cyclictest1080713-21/usr/sbin/munin00:25:123
96927699190,7cyclictest984075-21apt19:50:113
96927699190,7cyclictest1078519-21seq00:17:283
96927699190,1cyclictest3056-21PK-Backend23:27:223
96927699190,1cyclictest0-21swapper/321:34:413
96927699190,1cyclictest0-21swapper/320:36:553
96927699190,1cyclictest0-21swapper/300:30:093
96927699190,18cyclictest0-21swapper/323:59:133
96927699190,18cyclictest0-21swapper/322:25:033
96927299197,3cyclictest34-21ksoftirqd/220:45:202
96927299195,3cyclictest141rcu_preempt20:05:022
969272991917,1cyclictest0-21swapper/223:01:222
969272991917,1cyclictest0-21swapper/219:24:112
969272991911,6cyclictest2399-21gdbus21:18:032
969272991911,6cyclictest1006901-21cat20:55:132
969272991911,2cyclictest34-21ksoftirqd/219:45:122
96927299190,1cyclictest0-21swapper/223:41:172
96927299190,1cyclictest0-21swapper/222:39:122
969266991917,1cyclictest3056-21PK-Backend22:54:531
969266991917,1cyclictest3056-21PK-Backend21:39:161
969266991917,1cyclictest0-21swapper/122:10:051
969266991917,1cyclictest0-21swapper/119:37:491
969266991917,1cyclictest0-21swapper/100:25:451
96926699190,1cyclictest3056-21PK-Backend23:36:031
96926699190,1cyclictest3056-21PK-Backend21:56:321
96926699190,1cyclictest3056-21PK-Backend20:51:301
96926699190,1cyclictest3056-21PK-Backend19:59:591
96926699190,1cyclictest3056-21PK-Backend00:03:001
96926699190,1cyclictest0-21swapper/123:20:351
96926699190,1cyclictest0-21swapper/119:12:411
96926699190,18cyclictest3056-21PK-Backend23:45:501
96926699190,18cyclictest3056-21PK-Backend23:31:101
96926699190,18cyclictest3056-21PK-Backend23:31:101
96926699190,18cyclictest3056-21PK-Backend21:17:461
96926699190,18cyclictest0-21swapper/122:21:551
969262991912,5cyclictest1069780-21cron23:55:000
969262991912,5cyclictest1027856-21sed21:55:010
969262991911,6cyclictest436-20systemd-journal20:10:000
969262991911,6cyclictest1052456-21ls23:05:000
969262991911,5cyclictest1064602-21idleruntime-cro23:40:000
10552962190,3sleep0161rcuc/023:10:190
969276991816,1cyclictest3056-21PK-Backend23:06:063
969276991816,1cyclictest3056-21PK-Backend22:14:013
969276991816,1cyclictest0-21swapper/320:58:043
969276991816,1cyclictest0-21swapper/319:59:033
969276991813,4cyclictest42-21ksoftirqd/319:15:013
969276991813,3cyclictest1073539-21turbostat.cron00:05:003
969276991813,3cyclictest1029672-21apt22:00:113
96927699180,1cyclictest3056-21PK-Backend21:13:393
96927699180,16cyclictest9835112sleep319:46:293
96927299187,3cyclictest10793692/usr/sbin/munin00:20:132
96927299187,2cyclictest141rcu_preempt00:38:382
969272991816,1cyclictest0-21swapper/221:53:222
969272991812,5cyclictest34-21ksoftirqd/220:25:152
96927299180,1cyclictest0-21swapper/219:12:552
969266991816,1cyclictest3056-21PK-Backend23:51:301
969266991816,1cyclictest3056-21PK-Backend22:02:551
969266991816,1cyclictest3056-21PK-Backend21:06:041
969266991816,1cyclictest3056-21PK-Backend00:17:271
969266991816,1cyclictest3056-21PK-Backend00:08:211
969266991816,1cyclictest0-21swapper/123:26:161
969266991815,2cyclictest3056-21PK-Backend21:51:331
969266991815,2cyclictest3056-21PK-Backend21:01:241
969266991811,5cyclictest1053722-21cat23:05:211
96926299182,11cyclictest1041800-21munin-run22:35:000
969262991814,3cyclictest1079177-21tr00:20:130
969262991813,3cyclictest1024211-21awk21:45:000
969262991812,5cyclictest13-21ksoftirqd/023:46:130
969262991811,5cyclictest436-20systemd-journal20:00:000
969262991811,5cyclictest13-21ksoftirqd/000:15:230
969262991811,5cyclictest1085746-21date00:40:010
969262991811,5cyclictest1005446-21cat20:50:180
96926299180,3cyclictest1045520-21perf22:45:000
969276991712,3cyclictest42-21ksoftirqd/319:25:013
969272991712,4cyclictest1030318-21tail22:00:172
969272991710,2cyclictest34-21ksoftirqd/222:15:032
969272991710,2cyclictest34-21ksoftirqd/220:44:372
969272991710,2cyclictest34-21ksoftirqd/220:44:372
969266991715,1cyclictest3056-21PK-Backend20:04:121
969266991715,1cyclictest0-21swapper/120:24:201
969266991715,1cyclictest0-21swapper/119:16:331
969266991714,2cyclictest3056-21PK-Backend20:56:101
96926299179,5cyclictest1013285-21apt-get21:15:020
96926299172,13cyclictest974339-21needreboot19:20:160
969262991715,1cyclictest0-21swapper/023:55:500
969262991713,3cyclictest994207-21sh20:20:000
969262991711,5cyclictest1036109-21ls22:15:210
969262991711,4cyclictest981089-21taskset19:40:160
969262991711,4cyclictest1016217-21needreboot21:20:180
969262991710,5cyclictest983333-21cat19:45:210
969262991710,5cyclictest1050707-21cron23:00:000
96926299170,15cyclictest988988-21cron20:05:000
96927699165,4cyclictest42-21ksoftirqd/322:24:463
96927299164,4cyclictest34-21ksoftirqd/200:25:392
969266991614,1cyclictest3056-21PK-Backend21:23:281
96926299169,6cyclictest1-21systemd22:55:000
96926299169,5cyclictest436-20systemd-journal21:09:290
96926299169,5cyclictest1056343-21grep23:15:130
96926299169,5cyclictest1007645-21cat20:55:220
969262991614,1cyclictest0-21swapper/020:32:450
969262991611,4cyclictest1009291-21sed21:00:220
96926299161,11cyclictest1066333-21munin-run23:45:000
969262991610,4cyclictest436-20systemd-journal21:30:000
969262991610,4cyclictest1028978-21grep21:55:240
96926299160,15cyclictest0-21swapper/020:12:310
139099160,3rtkit-daemon3056-21PK-Backend00:26:180
139099160,3rtkit-daemon0-21swapper/023:08:010
96926699158,5cyclictest1066321-21cron23:45:001
96926299159,4cyclictest1082953-21cut00:30:150
96926299159,4cyclictest1073978-21tr00:05:130
96926299159,4cyclictest1058592-21tail23:20:170
96926299159,4cyclictest1025087-21grep21:45:180
96926299152,11cyclictest1020890-21cpuspeed_turbos21:35:120
969262991513,1cyclictest0-21swapper/019:50:330
969262991511,3cyclictest1095-21runrttasks00:10:160
969262991511,3cyclictest1038571-21/usr/sbin/munin22:25:160
969262991511,3cyclictest1029729-21sort22:00:120
139099150,3rtkit-daemon3056-21PK-Backend22:33:271
969266991412,1cyclictest3056-21PK-Backend20:31:101
969262991412,1cyclictest0-21swapper/021:32:330
10615992140,4chrt3056-21PK-Backend23:30:120
10615992140,4chrt3056-21PK-Backend23:30:120
96926699139,3cyclictest3056-21PK-Backend22:26:291
96926699139,3cyclictest3056-21PK-Backend19:27:331
96926699136,5cyclictest984282-21cstates19:50:131
969266991311,1cyclictest3056-21PK-Backend23:58:271
96926299126,5cyclictest994956-21grep20:20:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional