You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 05:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Sep 23, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
165193826544,14sleep30-21swapper/319:07:413
165206326344,13sleep00-21swapper/019:09:190
165183326344,13sleep20-21swapper/219:06:182
165192926242,13sleep10-21swapper/119:07:341
17279312520,4sleep13056-21PK-Backend22:40:191
17227742460,4sleep00-21swapper/022:25:220
17365412450,3sleep20-21swapper/223:05:182
139099360,4rtkit-daemon1389-21rtkit-daemon20:48:161
1652269993015,12cyclictest1749898-21unin-run23:45:011
139099290,3rtkit-daemon1389-21rtkit-daemon21:36:491
1652269992812,8cyclictest26-21ksoftirqd/121:02:051
165226999270,3cyclictest26-21ksoftirqd/119:35:001
139099270,4rtkit-daemon1389-21rtkit-daemon21:39:380
165227199260,2cyclictest141rcu_preempt00:25:232
139099260,4rtkit-daemon1389-21rtkit-daemon20:29:210
165227599258,15cyclictest1-21systemd22:34:133
165227199258,15cyclictest1709233-21unin-run21:50:002
1652271992520,4cyclictest34-21ksoftirqd/200:10:002
1652271992519,4cyclictest1689409-21timerwakeupswit20:50:232
1652269992513,8cyclictest1758550-21cron00:10:001
17031732240,3chrt0-21swapper/021:30:200
165226999242,4cyclictest26-21ksoftirqd/123:40:011
1652269992412,10cyclictest1719858-21cron22:20:001
1652269992412,10cyclictest1719858-21cron22:20:001
165226299240,22cyclictest1690133-21apt20:55:100
165226299240,22cyclictest1389-21rtkit-daemon22:43:270
16831412230,10sleep234-21ksoftirqd/220:35:002
1652275992320,2cyclictest3056-21PK-Backend19:57:553
1652275992320,1cyclictest3056-21PK-Backend19:40:043
1652275992316,5cyclictest883-21dbus-daemon00:08:033
165227599231,19cyclictest0-21swapper/300:10:143
165227199230,2cyclictest141rcu_preempt00:05:002
1652269992320,2cyclictest0-21swapper/121:45:281
1652269992319,3cyclictest251rcuc/123:09:571
1652269992319,3cyclictest251rcuc/121:29:471
1652269992312,3cyclictest26-21ksoftirqd/119:15:471
165226999231,15cyclictest1095-21runrttasks23:00:121
1652269992310,11cyclictest1696784-21cron21:14:591
1652269992310,11cyclictest1696784-21cron21:14:591
165226999230,3cyclictest26-21ksoftirqd/100:15:001
17508972220,4sleep33056-21PK-Backend23:45:213
17444282220,4chrt1744413-21munin-run23:30:002
1652275992220,1cyclictest0-21swapper/322:39:303
1652275992215,5cyclictest1710577-21sshd21:50:393
165227199228,2cyclictest141rcu_preempt22:30:002
1652271992216,4cyclictest34-21ksoftirqd/220:20:012
165226999223,3cyclictest26-21ksoftirqd/120:30:011
1652269992219,2cyclictest251rcuc/122:21:201
1652269992219,2cyclictest251rcuc/119:40:351
165226999221,6cyclictest26-21ksoftirqd/119:25:021
1652269992211,3cyclictest26-21ksoftirqd/121:51:501
165226999220,8cyclictest26-21ksoftirqd/100:35:131
165226999220,6cyclictest1684665-21perf20:40:001
139099220,3rtkit-daemon1389-21rtkit-daemon20:30:360
1652275992119,1cyclictest3056-21PK-Backend23:41:223
1652275992119,1cyclictest3056-21PK-Backend20:44:523
1652275992119,1cyclictest0-21swapper/323:17:363
1652275992119,1cyclictest0-21swapper/321:00:073
1652275992119,1cyclictest0-21swapper/320:21:273
1652275992119,1cyclictest0-21swapper/320:17:143
1652275992119,1cyclictest0-21swapper/300:38:543
1652275992119,1cyclictest0-21swapper/300:01:233
1652275992118,2cyclictest3056-21PK-Backend21:46:443
1652275992118,2cyclictest3056-21PK-Backend21:31:313
1652275992118,2cyclictest3056-21PK-Backend00:30:223
1652275992113,6cyclictest1-21systemd19:15:033
165227599210,20cyclictest0-21swapper/323:31:283
165227599210,20cyclictest0-21swapper/320:30:063
165227599210,19cyclictest3056-21PK-Backend23:53:053
165227599210,19cyclictest3056-21PK-Backend22:57:223
165227599210,19cyclictest3056-21PK-Backend19:27:063
165227199217,3cyclictest16940942kernelversion21:05:192
165227199217,13cyclictest1657919-21unin-run19:25:012
1652271992118,2cyclictest3056-21PK-Backend20:00:352
1652271992114,6cyclictest1709860-21fw_packets21:50:152
1652271992114,6cyclictest1-21systemd23:40:002
1652271992113,6cyclictest1692726-21python321:00:202
1652271992113,2cyclictest34-21ksoftirqd/222:45:232
1652271992111,9cyclictest1769765-21idleruntime-cro00:40:002
1652271992111,3cyclictest34-21ksoftirqd/219:15:162
1652271992110,10cyclictest1755109-40.mlocate00:00:012
165227199210,20cyclictest0-21swapper/223:41:252
165227199210,20cyclictest0-21swapper/221:39:182
165227199210,20cyclictest0-21swapper/219:39:392
165227199210,20cyclictest0-21swapper/200:30:052
165226999217,2cyclictest141rcu_preempt20:18:501
165226999213,4cyclictest26-21ksoftirqd/121:00:011
165226999213,10cyclictest26-21ksoftirqd/119:47:101
1652269992119,1cyclictest3056-21PK-Backend00:02:351
1652269992119,1cyclictest0-21swapper/122:54:551
1652269992118,2cyclictest251rcuc/121:31:551
1652269992118,2cyclictest251rcuc/120:03:211
1652269992118,2cyclictest251rcuc/100:31:001
165226999211,7cyclictest26-21ksoftirqd/123:58:331
165226999211,7cyclictest26-21ksoftirqd/123:35:001
165226999211,7cyclictest26-21ksoftirqd/119:15:001
1652269992117,3cyclictest251rcuc/121:22:251
1652269992117,3cyclictest251rcuc/119:38:151
1652269992114,4cyclictest1674002-21idleruntime-cro20:10:001
1652269992113,6cyclictest1721872-21apt22:25:121
165226999210,7cyclictest1717545-21/usr/sbin/munin22:10:261
1652262992119,1cyclictest0-21swapper/021:54:050
1652262992115,4cyclictest1695060-21sh21:10:000
1652262992114,5cyclictest1737490-21basename23:09:590
1652262992112,7cyclictest1741952-21ntp_states23:20:210
139099210,17rtkit-daemon1788643-21eadedCompositor22:17:400
139099210,17rtkit-daemon1788643-21eadedCompositor22:17:400
165227599202,11cyclictest1095-21runrttasks23:20:123
1652275992018,1cyclictest3056-21PK-Backend21:26:583
1652275992018,1cyclictest3056-21PK-Backend19:37:143
1652275992018,1cyclictest0-21swapper/323:25:133
1652275992018,1cyclictest0-21swapper/323:14:113
1652275992013,5cyclictest1728062-21sed22:40:203
165227599200,1cyclictest0-21swapper/319:20:023
165227599200,19cyclictest3056-21PK-Backend23:37:433
165227599200,19cyclictest3056-21PK-Backend23:09:593
165227599200,19cyclictest3056-21PK-Backend23:00:463
165227599200,19cyclictest3056-21PK-Backend21:24:003
165227599200,19cyclictest0-21swapper/322:08:483
165227599200,19cyclictest0-21swapper/322:03:143
165227599200,19cyclictest0-21swapper/320:38:593
165227599200,19cyclictest0-21swapper/319:48:353
165227599200,18cyclictest1694181-21memory21:05:203
165227199206,3cyclictest141rcu_preempt19:34:592
1652271992018,1cyclictest0-21swapper/222:55:082
1652271992018,1cyclictest0-21swapper/221:17:542
1652271992018,1cyclictest0-21swapper/219:41:422
1652271992012,2cyclictest34-21ksoftirqd/222:00:132
1652271992011,3cyclictest34-21ksoftirqd/219:50:132
165227199200,19cyclictest0-21swapper/223:47:542
165227199200,19cyclictest0-21swapper/223:30:312
165227199200,19cyclictest0-21swapper/223:04:352
165227199200,19cyclictest0-21swapper/222:39:412
165227199200,19cyclictest0-21swapper/221:41:402
165227199200,19cyclictest0-21swapper/221:10:252
165227199200,19cyclictest0-21swapper/221:10:252
165227199200,19cyclictest0-21swapper/219:47:382
165227199200,19cyclictest0-21swapper/219:25:262
165226999206,6cyclictest1763899-21chrt00:20:231
165226999203,7cyclictest26-21ksoftirqd/123:11:411
165226999203,5cyclictest26-21ksoftirqd/122:45:181
165226999202,7cyclictest26-21ksoftirqd/119:50:241
165226999202,6cyclictest1674068-21kworker/1:120:31:541
165226999202,4cyclictest26-21ksoftirqd/123:20:111
165226999201,8cyclictest26-21ksoftirqd/122:05:131
165226999201,7cyclictest26-21ksoftirqd/123:30:001
1652269992017,2cyclictest251rcuc/123:46:091
1652269992017,2cyclictest251rcuc/121:17:171
1652269992017,2cyclictest251rcuc/120:50:021
165226999201,6cyclictest26-21ksoftirqd/123:00:001
1652269992016,3cyclictest251rcuc/100:28:391
1652269992015,4cyclictest26-21ksoftirqd/121:40:411
165226999200,1cyclictest3056-21PK-Backend21:55:471
1652262992018,1cyclictest0-21swapper/022:00:270
1652262992018,1cyclictest0-21swapper/019:49:270
1652262992018,1cyclictest0-21swapper/000:21:410
1652262992017,2cyclictest3056-21PK-Backend22:45:290
1652262992015,4cyclictest13-21ksoftirqd/000:00:010
1652262992015,3cyclictest1702006-21sh21:30:010
1652262992013,5cyclictest1657868-21cron19:24:590
165226299200,1cyclictest0-21swapper/020:05:430
165226299200,19cyclictest0-21swapper/023:10:430
165226299200,19cyclictest0-21swapper/022:38:390
165226299200,19cyclictest0-21swapper/022:22:320
165226299200,19cyclictest0-21swapper/021:56:290
165226299200,19cyclictest0-21swapper/021:46:210
165226299200,19cyclictest0-21swapper/021:11:470
165226299200,19cyclictest0-21swapper/021:11:470
165226299200,19cyclictest0-21swapper/020:53:350
165226299200,19cyclictest0-21swapper/020:49:020
165226299200,19cyclictest0-21swapper/020:43:070
165226299200,19cyclictest0-21swapper/020:15:290
165226299200,19cyclictest0-21swapper/019:44:340
165226299200,19cyclictest0-21swapper/000:09:450
139099200,3rtkit-daemon1389-21rtkit-daemon22:09:510
139099200,16rtkit-daemon1652258-21cyclictest23:03:090
1652275991917,1cyclictest0-21swapper/322:45:323
1652275991917,1cyclictest0-21swapper/320:27:293
1652275991917,1cyclictest0-21swapper/320:05:053
1652275991917,1cyclictest0-21swapper/300:15:563
1652275991916,2cyclictest3056-21PK-Backend20:00:183
1652275991914,4cyclictest978-21snmpd19:14:523
1652275991912,5cyclictest1690757-21cat20:55:173
165227599190,1cyclictest3056-21PK-Backend21:15:553
165227599190,1cyclictest3056-21PK-Backend19:54:303
165227599190,1cyclictest3056-21PK-Backend19:30:443
165227599190,1cyclictest0-21swapper/323:58:323
165227599190,17cyclictest436-20systemd-journal22:10:393
165227199195,12cyclictest1753327-21sh23:55:002
165227199191,8cyclictest141rcu_preempt00:10:192
1652271991917,1cyclictest0-21swapper/223:13:132
1652271991917,1cyclictest0-21swapper/222:05:342
1652271991917,1cyclictest0-21swapper/221:22:272
1652271991917,1cyclictest0-21swapper/220:44:162
1652271991917,1cyclictest0-21swapper/220:14:342
1652271991916,2cyclictest0-21swapper/222:52:312
1652271991912,5cyclictest34-21ksoftirqd/220:59:502
1652271991912,5cyclictest1683645-21cat20:35:142
1652271991912,2cyclictest34-21ksoftirqd/222:15:152
1652271991912,2cyclictest34-21ksoftirqd/222:15:152
1652271991911,2cyclictest34-21ksoftirqd/223:15:162
1652271991911,2cyclictest34-21ksoftirqd/222:45:002
1652271991911,2cyclictest34-21ksoftirqd/220:45:202
1652271991911,2cyclictest34-21ksoftirqd/220:25:182
1652271991910,6cyclictest34-21ksoftirqd/219:55:132
1652271991910,3cyclictest34-21ksoftirqd/223:20:112
165227199190,1cyclictest0-21swapper/222:12:442
165227199190,1cyclictest0-21swapper/221:27:002
165227199190,1cyclictest0-21swapper/200:22:072
165227199190,18cyclictest0-21swapper/220:21:232
165226999195,4cyclictest26-21ksoftirqd/122:00:201
165226999193,5cyclictest26-21ksoftirqd/120:12:381
1652269991916,2cyclictest251rcuc/120:41:151
1652269991915,3cyclictest251rcuc/123:17:561
1652269991914,4cyclictest26-21ksoftirqd/119:25:011
1652269991910,3cyclictest26-21ksoftirqd/121:08:351
1652262991917,1cyclictest3056-21PK-Backend19:15:400
1652262991917,1cyclictest0-21swapper/023:29:220
1652262991917,1cyclictest0-21swapper/023:19:090
1652262991917,1cyclictest0-21swapper/022:58:540
1652262991917,1cyclictest0-21swapper/021:01:200
1652262991917,1cyclictest0-21swapper/000:32:160
1652262991917,1cyclictest0-21swapper/000:00:520
1652262991912,6cyclictest1731558-21awk22:50:200
1652262991912,5cyclictest1746857-21expr23:35:120
165226299190,1cyclictest0-21swapper/023:51:470
165226299190,1cyclictest0-21swapper/023:42:410
165226299190,1cyclictest0-21swapper/022:32:040
165226299190,1cyclictest0-21swapper/020:35:520
165226299190,1cyclictest0-21swapper/020:01:450
165226299190,1cyclictest0-21swapper/019:50:280
165226299190,1cyclictest0-21swapper/019:36:030
165226299190,18cyclictest0-21swapper/023:48:080
165226299190,18cyclictest0-21swapper/021:16:540
165226299190,18cyclictest0-21swapper/019:59:200
165226299190,18cyclictest0-21swapper/000:37:570
165226299190,18cyclictest0-21swapper/000:17:500
165227599183,13cyclictest1765406-21python300:25:223
1652275991816,1cyclictest3056-21PK-Backend22:53:573
1652275991816,1cyclictest3056-21PK-Backend21:11:363
1652275991816,1cyclictest3056-21PK-Backend21:11:363
1652275991816,1cyclictest0-21swapper/300:21:373
1652275991811,6cyclictest1718399-21awk22:15:103
1652275991811,6cyclictest1718399-21awk22:15:103
1652275991810,6cyclictest1704994-21ntpq21:35:183
165227199185,3cyclictest141rcu_preempt21:30:172
1652271991816,1cyclictest0-21swapper/200:16:262
165226999183,13cyclictest1753321-21cron23:55:001
165226999182,5cyclictest3056-21PK-Backend00:15:161
1652269991815,2cyclictest251rcuc/120:20:531
1652269991814,3cyclictest251rcuc/122:32:001
1652269991810,3cyclictest26-21ksoftirqd/122:36:201
165226999180,6cyclictest26-21ksoftirqd/119:56:491
1652262991816,1cyclictest0-21swapper/023:33:010
1652262991811,5cyclictest1661947-21wc19:35:010
1652262991810,6cyclictest1658371-21seq19:25:010
1652262991810,2cyclictest13-21ksoftirqd/022:15:000
165226299180,1cyclictest0-21swapper/021:43:580
165226299180,1cyclictest0-21swapper/000:25:330
1652275991715,1cyclictest3056-21PK-Backend21:40:363
1652275991715,1cyclictest0-21swapper/320:53:103
165227599171,2cyclictest0-21swapper/321:58:393
165227199179,5cyclictest34-21ksoftirqd/222:21:032
1652271991712,3cyclictest1724067-21if_err_enp2s022:30:182
165227199171,14cyclictest1673200-21grep20:05:172
165227199171,12cyclictest1711464-21apt21:55:112
165226299177,4cyclictest141rcu_preempt19:15:010
1652262991715,1cyclictest0-21swapper/000:10:460
1652262991711,5cyclictest1675186-21cat20:10:200
139099170,14rtkit-daemon0-21swapper/021:21:240
165227599169,5cyclictest1720155-21grep22:20:183
165227599161,13cyclictest1723292-21sshd22:29:413
165227599160,2cyclictest3056-21PK-Backend20:45:193
1652271991613,2cyclictest1095-21runrttasks19:12:502
1652262991610,4cyclictest1678511-21grep20:20:170
165227599139,3cyclictest1674240-21/usr/sbin/munin20:10:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional