You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 07:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Jun 24, 2026 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
60913221280,3sleep0609130-21wc20:00:350
58929127252,14sleep00-21swapper/019:05:370
58945826545,14sleep30-21swapper/319:07:453
58936726545,13sleep10-21swapper/119:06:351
58929326545,14sleep20-21swapper/219:05:402
6385852500,1sleep10-21swapper/121:25:171
6805852490,3sleep10-21swapper/123:25:181
6577282430,2sleep00-21swapper/022:20:130
58977699328,2cyclictest597186-21apt-config19:30:002
589776993213,4cyclictest34-21ksoftirqd/221:15:012
58976899322,15cyclictest662632-21apt22:35:110
58976899322,15cyclictest662632-21apt22:35:110
58976899314,12cyclictest700766-21latency_hist00:25:010
589776993013,4cyclictest34-21ksoftirqd/221:05:002
589768993015,13cyclictest610787-21ntp_states20:05:180
58976899301,14cyclictest0-21swapper/023:00:000
58977699288,11cyclictest34-21ksoftirqd/223:05:012
58977699287,10cyclictest34-21ksoftirqd/220:00:012
58977699285,4cyclictest34-21ksoftirqd/220:20:002
58976899285,7cyclictest1-21systemd19:40:000
58976899285,7cyclictest1-21systemd19:39:590
58976899283,7cyclictest660627-21munin-run22:30:000
58977699272,7cyclictest34-21ksoftirqd/222:50:002
58977699270,8cyclictest701696-21strings00:25:182
58976899272,14cyclictest631371-21cstates21:05:140
58976899272,13cyclictest690373-21systemctl23:55:010
58976899272,10cyclictest698038-21irqstats00:15:170
58977699265,10cyclictest34-21ksoftirqd/222:15:012
589776992611,3cyclictest34-21ksoftirqd/221:42:562
58977699260,7cyclictest34-21ksoftirqd/220:15:002
58976899262,15cyclictest623708-21munin-run20:45:000
58976899262,15cyclictest623708-21munin-run20:45:000
58976899261,13cyclictest606319-21munin-run19:54:590
58976899260,8cyclictest0-21swapper/022:55:010
58977699256,7cyclictest34-21ksoftirqd/223:40:002
58977699255,11cyclictest34-21ksoftirqd/223:10:002
58977699254,6cyclictest34-21ksoftirqd/222:25:132
58977699250,3cyclictest621955-21cron20:39:592
58977299250,24cyclictest614124-21ls20:15:141
58976899253,8cyclictest677982-21systemd-run23:20:010
58976899253,10cyclictest602856-21munin-run19:45:000
58976899251,8cyclictest679714-21systemd-run23:25:010
58976899251,13cyclictest0-21swapper/021:00:220
58976899251,13cyclictest0-21swapper/020:00:000
58977699246,4cyclictest34-21ksoftirqd/219:20:002
58977699243,6cyclictest34-21ksoftirqd/222:40:252
58977699242,7cyclictest34-21ksoftirqd/221:07:422
58977699242,2cyclictest141rcu_preempt23:00:012
589776992413,4cyclictest34-21ksoftirqd/219:44:592
589776992413,2cyclictest704219-21sh00:34:592
589776992412,4cyclictest34-21ksoftirqd/219:35:322
589776992412,4cyclictest34-21ksoftirqd/219:35:312
58976899245,7cyclictest683494-21gpgv23:35:010
58976899244,8cyclictest597524-21gpgconf19:30:000
58976899242,8cyclictest638456-21df21:25:140
58976899242,7cyclictest634533-21cat21:15:010
58976899241,7cyclictest0-21swapper/023:10:220
58976899241,7cyclictest0-21swapper/021:45:120
589768992413,9cyclictest652064-21cat22:05:010
58976899240,15cyclictest0-21swapper/020:15:010
58977699239,4cyclictest34-21ksoftirqd/221:55:192
58977699236,2cyclictest141rcu_preempt23:55:172
58977699233,7cyclictest34-21ksoftirqd/220:50:182
589776992319,3cyclictest34-21ksoftirqd/223:20:002
58977699231,8cyclictest34-21ksoftirqd/221:20:012
589776992314,5cyclictest652329-21sort22:05:112
589776992312,3cyclictest34-21ksoftirqd/222:03:562
589776992311,3cyclictest141rcu_preempt20:56:432
589772992315,7cyclictest703837-21/usr/sbin/munin00:30:241
58977299230,4cyclictest600642-21runrttasks19:37:301
58977299230,4cyclictest600642-21runrttasks19:37:301
58976899233,7cyclictest621980-21munin-run20:40:000
58976899233,6cyclictest627195-21unin-run20:55:000
58976899232,6cyclictest657172-21munin-run22:20:000
58976899230,9cyclictest679930-21/usr/sbin/munin23:25:170
58976899230,8cyclictest0-21swapper/000:10:200
58976899230,10cyclictest0-21swapper/023:10:000
58977899220,21cyclictest3056-21PK-Backend20:24:323
58977899220,20cyclictest3056-21PK-Backend21:20:423
589776992219,2cyclictest34-21ksoftirqd/200:18:122
589776992211,3cyclictest141rcu_preempt21:36:092
589776992211,3cyclictest141rcu_preempt20:42:412
589776992211,3cyclictest141rcu_preempt20:42:402
58977699220,21cyclictest662632-21apt22:35:112
58977699220,21cyclictest662632-21apt22:35:102
58977699220,20cyclictest886-21NetworkManager23:50:002
58977299220,21cyclictest0-21swapper/100:38:101
58977299220,21cyclictest0-21swapper/100:38:091
58976899229,10cyclictest883-21dbus-daemon20:20:010
58976899222,7cyclictest667563-21munin-run22:50:010
589768992220,1cyclictest0-21swapper/023:30:320
589768992220,1cyclictest0-21swapper/021:30:070
58976899221,8cyclictest649507-21ntp_states21:55:230
58976899221,8cyclictest0-21swapper/019:45:110
589768992214,5cyclictest704233-21turbostat.cron00:34:590
589768992211,7cyclictest701029-21apt00:25:110
58976899221,13cyclictest0-21swapper/019:28:380
58976899220,21cyclictest0-21swapper/023:40:390
58976899220,13cyclictest0-21swapper/022:15:010
589778992119,1cyclictest3056-21PK-Backend20:53:313
589778992119,1cyclictest3056-21PK-Backend19:59:383
589778992119,1cyclictest3056-21PK-Backend00:11:513
58977899210,20cyclictest0-21swapper/319:43:433
58977899210,1cyclictest3056-21PK-Backend20:40:213
58977899210,1cyclictest3056-21PK-Backend20:40:213
58977899210,1cyclictest141rcu_preempt22:15:013
58977899210,19cyclictest0-21swapper/300:07:383
58977699217,3cyclictest141rcu_preempt20:01:312
589776992119,1cyclictest0-21swapper/221:29:272
589776992113,6cyclictest679674-21cron23:25:002
589776992111,2cyclictest141rcu_preempt21:21:552
589776992111,2cyclictest141rcu_preempt00:10:512
589776992110,3cyclictest141rcu_preempt22:23:292
589776992110,3cyclictest141rcu_preempt21:34:572
589776992110,3cyclictest141rcu_preempt20:26:372
589776992110,3cyclictest141rcu_preempt20:20:112
589776992110,3cyclictest141rcu_preempt19:53:332
589776992110,3cyclictest141rcu_preempt19:32:082
58977699210,8cyclictest34-21ksoftirqd/219:25:012
58977699210,20cyclictest0-21swapper/222:16:192
58977699210,20cyclictest0-21swapper/219:14:212
58977699210,11cyclictest34-21ksoftirqd/223:55:002
58977299217,7cyclictest141rcu_preempt19:45:111
589772992114,5cyclictest676883-21smartctl23:15:151
589772992113,6cyclictest436-20systemd-journal22:30:001
589772992112,7cyclictest436-20systemd-journal22:10:001
58977299210,20cyclictest0-21swapper/122:42:041
589768992119,1cyclictest0-21swapper/023:03:350
589768992119,1cyclictest0-21swapper/019:12:180
589768992119,1cyclictest0-21swapper/000:07:160
58976899211,8cyclictest0-21swapper/022:40:120
58976899211,8cyclictest0-21swapper/022:05:200
58976899211,8cyclictest0-21swapper/021:40:180
58976899210,7cyclictest0-21swapper/023:55:130
58976899210,2cyclictest13-21ksoftirqd/000:04:460
58976899210,20cyclictest0-21swapper/000:38:190
58976899210,20cyclictest0-21swapper/000:38:180
58977899207,8cyclictest42-21ksoftirqd/322:50:133
589778992018,1cyclictest3056-21PK-Backend22:27:073
589778992018,1cyclictest3056-21PK-Backend21:41:523
589778992018,1cyclictest3056-21PK-Backend19:36:403
589778992018,1cyclictest3056-21PK-Backend19:36:393
589778992018,1cyclictest3056-21PK-Backend19:33:083
589778992017,2cyclictest3056-21PK-Backend23:18:323
58977899200,2cyclictest42-21ksoftirqd/321:53:343
58977899200,2cyclictest141rcu_preempt21:05:013
58977899200,1cyclictest3056-21PK-Backend22:21:263
58977899200,1cyclictest3056-21PK-Backend19:48:433
58977899200,1cyclictest141rcu_preempt19:55:003
58977899200,1cyclictest0-21swapper/319:20:243
58977899200,19cyclictest3056-21PK-Backend22:36:123
58977899200,19cyclictest3056-21PK-Backend22:36:123
58977899200,19cyclictest3056-21PK-Backend22:31:063
58977899200,19cyclictest3056-21PK-Backend22:31:063
58977899200,19cyclictest3056-21PK-Backend20:28:513
58977899200,19cyclictest3056-21PK-Backend19:18:563
58977899200,19cyclictest3056-21PK-Backend00:36:383
58977899200,19cyclictest3056-21PK-Backend00:36:383
58977699206,8cyclictest34-21ksoftirqd/200:20:202
58977699204,9cyclictest34-21ksoftirqd/222:35:002
58977699204,9cyclictest34-21ksoftirqd/222:34:592
58977699203,9cyclictest34-21ksoftirqd/223:42:282
58977699203,9cyclictest34-21ksoftirqd/223:30:022
58977699203,4cyclictest34-21ksoftirqd/222:45:122
58977699202,6cyclictest34-21ksoftirqd/200:00:422
589776992018,1cyclictest0-21swapper/221:51:312
589776992018,1cyclictest0-21swapper/200:05:212
589776992010,3cyclictest141rcu_preempt23:11:332
589776992010,2cyclictest141rcu_preempt23:27:302
589776992010,2cyclictest141rcu_preempt20:30:462
58977299205,13cyclictest648526-21unin-run21:55:011
589772992018,1cyclictest0-21swapper/123:46:181
589772992018,1cyclictest0-21swapper/120:07:311
589772992012,6cyclictest608896-21cat20:00:311
589772992012,6cyclictest594792-21python319:20:201
58977299200,1cyclictest0-21swapper/123:09:281
58977299200,19cyclictest0-21swapper/123:21:161
58977299200,19cyclictest0-21swapper/122:20:001
58977299200,19cyclictest0-21swapper/121:32:281
58977299200,19cyclictest0-21swapper/121:13:071
58977299200,19cyclictest0-21swapper/121:08:341
58977299200,19cyclictest0-21swapper/120:33:341
58977299200,19cyclictest0-21swapper/119:31:561
58976899207,11cyclictest615458-21cstates20:20:150
58976899203,5cyclictest0-21swapper/019:15:020
589768992018,1cyclictest0-21swapper/023:45:270
58976899200,19cyclictest0-21swapper/020:33:010
58976899200,17cyclictest883-21dbus-daemon20:29:590
58977899196,9cyclictest42-21ksoftirqd/300:00:133
58977899195,7cyclictest141rcu_preempt23:35:113
58977899195,2cyclictest141rcu_preempt21:15:013
58977899195,2cyclictest141rcu_preempt20:20:013
589778991917,1cyclictest3056-21PK-Backend22:15:443
589778991917,1cyclictest3056-21PK-Backend20:47:313
589778991917,1cyclictest3056-21PK-Backend20:03:093
589778991917,1cyclictest0-21swapper/323:54:283
589778991917,1cyclictest0-21swapper/319:11:243
58977899190,3cyclictest141rcu_preempt00:29:593
58977899190,1cyclictest3056-21PK-Backend23:27:173
58977899190,1cyclictest3056-21PK-Backend22:09:303
58977899190,1cyclictest3056-21PK-Backend21:59:493
58977899190,1cyclictest3056-21PK-Backend20:33:103
58977899190,1cyclictest3056-21PK-Backend00:18:213
58977899190,1cyclictest141rcu_preempt21:05:023
58977899190,1cyclictest0-21swapper/323:58:403
58977899190,18cyclictest3056-21PK-Backend21:39:363
58977899190,18cyclictest3056-21PK-Backend21:27:523
58977899190,18cyclictest3056-21PK-Backend20:12:423
58977699198,3cyclictest141rcu_preempt20:45:542
58977699196,3cyclictest34-21ksoftirqd/219:47:472
589776991917,1cyclictest0-21swapper/220:06:052
589776991917,1cyclictest0-21swapper/200:38:202
589776991917,1cyclictest0-21swapper/200:38:202
58977699190,6cyclictest645730-21/usr/sbin/munin21:45:162
589772991917,1cyclictest0-21swapper/123:55:171
589772991917,1cyclictest0-21swapper/123:52:121
589772991917,1cyclictest0-21swapper/123:01:511
589772991916,2cyclictest0-21swapper/123:35:031
589772991913,5cyclictest686057-21needreboot23:40:171
589772991912,6cyclictest694577-21awk00:05:211
589772991912,5cyclictest620810-21expr20:35:141
589772991911,6cyclictest655014-21sed22:10:201
58977299190,1cyclictest0-21swapper/121:19:221
58977299190,1cyclictest0-21swapper/120:14:071
58977299190,1cyclictest0-21swapper/119:50:541
58977299190,1cyclictest0-21swapper/100:21:191
58977299190,18cyclictest0-21swapper/121:24:301
58977299190,18cyclictest0-21swapper/120:22:581
58977299190,18cyclictest0-21swapper/119:42:431
58977299190,18cyclictest0-21swapper/119:18:581
58976899193,8cyclictest0-21swapper/021:20:220
589768991914,3cyclictest886-21NetworkManager19:25:010
58977899188,4cyclictest141rcu_preempt20:05:173
58977899186,8cyclictest42-21ksoftirqd/321:45:123
589778991816,1cyclictest3056-21PK-Backend22:40:113
589778991814,3cyclictest42-21ksoftirqd/322:50:013
58977899180,7cyclictest700562-21chrt00:22:393
58977899180,4cyclictest3056-21PK-Backend19:29:593
58977899180,1cyclictest3056-21PK-Backend20:39:123
58977299189,7cyclictest883-21dbus-daemon21:45:011
589772991816,1cyclictest0-21swapper/121:59:391
589772991814,3cyclictest2156-21gnome-shell00:27:091
589772991812,4cyclictest26-21ksoftirqd/120:30:001
589772991810,6cyclictest669925-21df22:55:131
58977299180,1cyclictest0-21swapper/123:30:501
58977299180,1cyclictest0-21swapper/121:04:361
58976899182,12cyclictest641825-21apt21:35:130
58976899180,6cyclictest635897-21sshd21:17:410
58976899180,1cyclictest0-21swapper/022:30:170
58976899180,1cyclictest0-21swapper/022:30:170
58977899178,3cyclictest141rcu_preempt23:50:013
58977899177,4cyclictest141rcu_preempt21:20:013
58977899177,3cyclictest141rcu_preempt23:30:173
58977899177,2cyclictest141rcu_preempt23:40:113
58977899175,3cyclictest141rcu_preempt22:00:553
589778991715,1cyclictest3056-21PK-Backend20:55:143
58977899171,2cyclictest141rcu_preempt00:34:593
58977299179,6cyclictest693108-21sshd00:00:201
58977299179,6cyclictest657895-21cat22:20:171
58977299179,6cyclictest650889-21fw_packets22:00:141
58977299179,6cyclictest622605-21df_inode20:40:131
58977299179,6cyclictest622605-21df_inode20:40:131
589772991713,3cyclictest596362-21netstat19:25:191
589772991712,4cyclictest26-21ksoftirqd/120:55:141
589772991712,3cyclictest625002-21timerwakeupswit20:45:221
589772991711,5cyclictest643316-21systemd-run21:40:001
589772991710,5cyclictest663157-21cat22:35:171
589772991710,5cyclictest663157-21cat22:35:161
58976899179,6cyclictest648088-21timerwakeupswit21:50:360
589768991710,5cyclictest624835-21smartctl20:45:210
58977899168,2cyclictest141rcu_preempt23:10:023
58977899166,9cyclictest671941-21needreboot23:00:173
589778991614,1cyclictest3056-21PK-Backend22:58:583
58977299169,5cyclictest696062-21expr00:10:121
58977299162,5cyclictest3056-21PK-Backend22:50:221
589772991614,1cyclictest0-21swapper/119:57:021
58977299161,11cyclictest674759-21apt23:10:131
589772991610,5cyclictest627197-21systemd20:55:011
589772991610,5cyclictest591365-21seq19:14:201
589772991610,5cyclictest26-21ksoftirqd/100:15:111
58976899167,7cyclictest627561-21grep20:55:140
58977899157,2cyclictest141rcu_preempt23:05:143
589778991511,2cyclictest42-21ksoftirqd/323:20:213
58977899150,2cyclictest141rcu_preempt21:32:443
58977299159,5cyclictest666656-21cat22:45:161
58977299159,4cyclictest643739-21grep21:40:121
58977299151,4cyclictest661079-21tr22:30:141
58977299151,4cyclictest661079-21tr22:30:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional