You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-29 - 00:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Sep 28, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1812929266,9sleep00-21swapper/007:09:550
1735528964,19sleep20-21swapper/207:05:242
1789328770,13sleep10-21swapper/107:06:511
1782328774,9sleep30-21swapper/307:05:563
119812540,1sleep013-21ksoftirqd/009:49:050
18226992827,1cyclictest0-21swapper/111:27:181
1822699251,23cyclictest0-21swapper/109:32:071
18221992422,2cyclictest13-21ksoftirqd/012:11:570
1822199240,23cyclictest0-21swapper/009:21:090
1822899230,20cyclictest0-21swapper/210:00:502
1822899230,1cyclictest0-21swapper/210:10:142
1822899220,21cyclictest0-21swapper/211:39:252
1822199210,3cyclictest0-21swapper/011:39:250
18228992019,1cyclictest0-21swapper/209:54:002
18226992018,1cyclictest0-21swapper/110:12:131
1822699200,2cyclictest0-21swapper/112:27:521
18231991917,1cyclictest0-21swapper/309:45:223
1822899190,1cyclictest0-21swapper/209:21:092
1822199190,18cyclictest0-21swapper/010:00:500
1823199182,6cyclictest3792-21fschecks_time11:40:153
1822899182,2cyclictest141rcu_preempt11:09:052
18228991816,1cyclictest0-21swapper/211:40:152
1822899181,3cyclictest141rcu_preempt11:16:392
1822699181,17cyclictest5006-21ssh09:45:021
1822199180,1cyclictest0-21swapper/011:17:130
1822199180,18cyclictest0-21swapper/010:18:160
1822199180,16cyclictest0-21swapper/010:53:330
1823199179,8cyclictest0-21swapper/309:20:373
18231991716,1cyclictest39-21ksoftirqd/311:19:273
1823199171,13cyclictest0-21swapper/310:25:153
1823199170,2cyclictest0-21swapper/309:36:363
1822899170,5cyclictest0-21swapper/212:05:282
1822899170,2cyclictest0-21swapper/212:32:542
1822899170,17cyclictest0-21swapper/210:53:322
1822899170,17cyclictest0-21swapper/209:29:552
18226991716,1cyclictest10813-21diskmemload10:41:461
1822699171,15cyclictest10813-21diskmemload10:51:571
18221991716,1cyclictest0-21swapper/011:03:550
1822199170,16cyclictest0-21swapper/009:29:550
1822199170,14cyclictest0-21swapper/011:59:280
1823199166,6cyclictest7972-21/usr/sbin/munin10:05:203
18231991615,1cyclictest0-21swapper/312:37:003
1823199160,3cyclictest0-21swapper/309:17:573
1823199160,1cyclictest0-21swapper/311:58:043
1823199160,15cyclictest0-21swapper/311:20:533
1822899162,1cyclictest10813-21diskmemload12:17:042
1822899160,16cyclictest0-21swapper/212:04:342
1822899160,15cyclictest0-21swapper/211:03:052
1822199168,2cyclictest13-21ksoftirqd/008:30:000
1822199165,7cyclictest10813-21diskmemload10:35:140
18221991614,1cyclictest0-21swapper/011:47:070
18221991613,2cyclictest13-21ksoftirqd/010:40:220
1822199160,1cyclictest0-21swapper/009:11:260
1822199160,16cyclictest0-21swapper/009:54:000
1822199160,14cyclictest0-21swapper/012:27:320
1822199160,0cyclictest0-21swapper/012:06:490
1823199151,1cyclictest0-21swapper/311:10:523
1823199150,2cyclictest0-21swapper/309:26:223
1822899158,1cyclictest11336-21apt-get09:10:142
1822899152,12cyclictest21106-21processes09:15:222
18228991514,1cyclictest0-21swapper/210:07:252
1822899150,2cyclictest0-21swapper/207:25:132
1822899150,14cyclictest0-21swapper/212:14:382
1822699157,7cyclictest16395-21wc10:50:001
1822699153,2cyclictest3056-21apt-get11:40:111
18226991515,0cyclictest0-21swapper/112:02:271
18226991513,1cyclictest10813-21diskmemload09:40:001
18226991510,4cyclictest25612-21apt-get07:25:131
1822699150,4cyclictest31129-21kworker/u8:011:46:531
1822699150,1cyclictest0-21swapper/111:01:501
1822199153,2cyclictest0-21swapper/008:10:140
18221991514,1cyclictest0-21swapper/010:47:060
18221991513,1cyclictest29372-21ssh12:34:170
1823199141,12cyclictest27114-21apache_processe11:35:123
1823199140,1cyclictest0-21swapper/309:52:233
1822899140,3cyclictest0-21swapper/211:30:082
1822899140,3cyclictest0-21swapper/210:20:542
1822899140,2cyclictest0-21swapper/210:42:192
1822899140,1cyclictest0-21swapper/211:59:282
1822899140,1cyclictest0-21swapper/209:35:062
1822899140,11cyclictest0-21swapper/210:49:132
1822699149,5cyclictest5602-21apache211:11:581
1822699149,3cyclictest29416-21ls07:30:251
1822699144,6cyclictest17281-21ls11:10:001
18226991412,1cyclictest38150irq/131-snd_hda10:04:481
18226991410,3cyclictest25110-21grep10:35:181
18226991410,3cyclictest1696-21cat11:20:011
1822699140,13cyclictest0-21swapper/109:18:301
1822699140,0cyclictest0-21swapper/111:51:111
1822199143,6cyclictest0-21swapper/011:20:130
18221991413,1cyclictest13-21ksoftirqd/011:26:310
18221991412,1cyclictest13-21ksoftirqd/009:31:490
1822199140,3cyclictest0-21swapper/011:50:480
1822199140,11cyclictest0-21swapper/010:10:150
1823199139,3cyclictest31672-21tr08:45:123
1823199135,8cyclictest11963-21sh09:10:163
18231991312,1cyclictest0-21swapper/310:35:153
18231991310,2cyclictest5814-21sh12:20:153
18231991310,2cyclictest31342-21grep10:00:163
1823199130,1cyclictest0-21swapper/311:01:053
1823199130,1cyclictest0-21swapper/307:45:123
1823199130,13cyclictest0-21swapper/312:11:153
1823199130,13cyclictest0-21swapper/311:08:223
1823199130,12cyclictest29277-21basename08:40:153
1822899139,3cyclictest20556-21cron07:15:002
1822899131,4cyclictest0-21swapper/211:20:132
1822899131,3cyclictest0-21swapper/210:25:132
18228991312,1cyclictest0-21swapper/212:21:262
1822899130,9cyclictest0-21swapper/209:49:082
1822899130,7cyclictest0-21swapper/210:37:462
1822899130,7cyclictest0-21swapper/210:17:312
1822899130,1cyclictest0-21swapper/209:42:502
1822899130,1cyclictest0-21swapper/209:42:492
1822899130,13cyclictest0-21swapper/211:47:062
1822699139,3cyclictest4330-21ssh09:25:141
1822699139,3cyclictest32640-21wc10:20:181
1822699139,3cyclictest28244-21tr09:20:161
1822699139,3cyclictest26829-21wc11:35:021
1822699137,5cyclictest4133-21tr08:55:141
1822699136,4cyclictest1490-21apt-get08:50:111
18226991310,2cyclictest8455-21grep10:25:171
18226991310,2cyclictest28542-21grep11:35:211
18226991310,2cyclictest20698-21perf12:10:001
18226991310,2cyclictest19353-21cron08:20:001
18226991310,2cyclictest11338-21grep08:00:211
1822699130,0cyclictest0-21swapper/110:07:161
1822199139,1cyclictest0-21swapper/010:59:250
1822199136,2cyclictest28810-21latency_hist09:40:000
1822199136,2cyclictest0-21swapper/008:50:110
1822199135,8cyclictest0-21swapper/010:20:540
18221991312,1cyclictest0-21swapper/010:28:120
1822199131,1cyclictest0-21swapper/010:30:120
1822199130,4cyclictest0-21swapper/011:40:190
1822199130,13cyclictest0-21swapper/011:11:060
1823199129,2cyclictest9416-21grep07:55:263
1823199129,2cyclictest30662-21sh09:40:233
1823199129,2cyclictest30662-21sh09:40:233
1823199129,2cyclictest29488-21ssh12:15:023
1823199129,2cyclictest28278-21cat08:35:263
1823199129,2cyclictest24930-21chrt12:30:493
1823199129,2cyclictest24232-21cat08:30:013
1823199129,2cyclictest228732sleep310:53:363
1823199129,2cyclictest2198-21grep08:50:163
1823199129,2cyclictest15192-21cat12:25:193
1823199129,2cyclictest12599-21wc11:45:183
1823199129,2cyclictest10898-21sh11:25:143
1823199128,3cyclictest25493-21grep10:55:143
1823199128,3cyclictest22093-21wc07:15:203
1823199128,3cyclictest14065-21cat08:05:273
18231991210,1cyclictest10813-21diskmemload10:42:283
1823199120,11cyclictest32004-21tr10:20:163
1823199120,0cyclictest0-21swapper/310:48:083
1822899129,2cyclictest7269-21chrt07:52:432
1822899129,2cyclictest5507-21grep08:55:252
1822899129,2cyclictest29900-21grep07:30:292
1822899129,2cyclictest24530-21grep08:30:142
1822899129,2cyclictest21941-21apt12:30:002
1822899129,2cyclictest18419-21ssh11:10:172
1822899122,8cyclictest27531-21ssh10:55:442
1822899121,5cyclictest0-21swapper/210:30:122
18228991210,2cyclictest32-21ksoftirqd/209:32:092
1822899120,12cyclictest0-21swapper/211:27:382
1822899120,11cyclictest16965-21cron08:15:002
1822899120,11cyclictest11903-21taskset08:01:012
1822699129,2cyclictest31645-21tail12:35:181
1822699129,2cyclictest29052-21cut11:55:161
1822699129,2cyclictest22524-21grep07:15:251
1822699128,3cyclictest12418-21wc09:10:191
1822699128,1cyclictest10813-21diskmemload09:44:471
1822699128,1cyclictest10813-21diskmemload09:44:471
1822199129,2cyclictest7936-21grep09:00:250
1822199129,2cyclictest6173-21cat12:20:160
1822199129,2cyclictest4256-21ssh12:38:290
1822199129,2cyclictest31812-21rm09:40:580
1822199129,2cyclictest31812-21rm09:40:580
1822199129,2cyclictest29826-21grep08:40:180
1822199129,2cyclictest27840-21cat07:30:000
1822199129,2cyclictest21486-21unixbench-2d09:15:260
1822199129,2cyclictest18297-21cat08:15:200
1822199129,2cyclictest1363-21ssh12:17:030
1822199129,2cyclictest11717-21sh10:07:490
1822199129,2cyclictest11573-21rm11:05:320
1822199125,6cyclictest537-21snmpd11:33:370
1822199120,11cyclictest19650-21apache_volume08:20:130
1823199119,1cyclictest19224-21grep07:10:183
1823199118,2cyclictest7472-21awk07:55:003
1823199118,2cyclictest25677-21rm09:56:443
1823199118,2cyclictest21054-21rm11:30:393
1823199118,2cyclictest20821-21expr11:50:153
1823199118,2cyclictest16462-21sh12:06:423
1823199118,2cyclictest15854-21tail10:10:153
1823199118,2cyclictest12539-21grep09:30:143
1823199117,3cyclictest752-21taskset07:40:123
1823199110,1cyclictest0-21swapper/312:00:433
1823199110,10cyclictest18948-21sh10:31:123
1822899119,2cyclictest959-21expr07:40:122
1822899119,1cyclictest8435-21grep07:55:172
1822899119,1cyclictest49382sleep207:49:042
1822899119,1cyclictest31043-21ssh12:35:152
1822899119,1cyclictest22817-21head09:55:162
1822899119,1cyclictest21836-21grep11:50:242
1822899119,1cyclictest1945-21expr08:50:132
1822899119,1cyclictest192232sleep208:18:242
1822899119,1cyclictest124542sleep208:05:052
1822899119,1cyclictest10713-21taskset09:09:032
1822899118,2cyclictest28836-21taskset08:39:402
1822899118,2cyclictest20950-21cut07:15:132
1822899110,10cyclictest8636-21date09:05:002
1822899110,10cyclictest325202sleep207:38:092
1822899110,10cyclictest25328-21chrt07:24:592
1822699118,2cyclictest8122-21taskset09:01:141
1822699118,2cyclictest5357-21ssh11:21:291
1822699118,2cyclictest32227-21grep07:35:261
1822699118,2cyclictest3206-21grep07:45:141
1822699118,2cyclictest25396-21chrt10:55:141
1822699118,2cyclictest25367-21head10:15:271
1822699118,2cyclictest25224-21chrt09:56:251
1822699118,2cyclictest24321-21cut07:20:181
1822699118,2cyclictest22668-21cut08:25:151
1822699118,2cyclictest21325-21sh12:10:111
1822699118,2cyclictest20555-21cron07:15:001
1822699118,2cyclictest20297-21cut08:20:171
1822699118,2cyclictest14159-21grep08:05:271
1822699118,2cyclictest13792-21cat12:25:001
1822699118,2cyclictest136062sleep109:50:111
1822699118,2cyclictest10914-21wc09:10:001
1822699118,2cyclictest1004-21ssh12:16:471
1822699117,2cyclictest17936-21head10:30:251
1822199119,2cyclictest29657-21sed07:30:280
1822199119,2cyclictest10017-21sendmail_mailst09:05:220
1822199119,1cyclictest5909-21rm12:00:230
1822199119,1cyclictest5842-21grep07:50:150
1822199119,1cyclictest28465-21grep08:35:270
1822199119,1cyclictest21871-21cat07:15:180
1822199119,1cyclictest12531-21tr08:05:150
1822199118,2cyclictest30994-21expr07:35:150
1822199118,2cyclictest26000-21grep08:30:260
1822199118,2cyclictest22871-21cut09:55:170
1822199118,2cyclictest1171-21taskset08:49:320
1822199112,5cyclictest225-21systemd-journal07:57:210
1823199108,1cyclictest28438-21grep07:30:183
1823199108,1cyclictest276562sleep307:29:243
1823199108,1cyclictest23629-21expr07:20:133
1823199108,1cyclictest11661-21expr08:00:253
1823199107,2cyclictest8668-21basename09:05:123
1823199107,2cyclictest32681-21sh07:40:003
1823199107,2cyclictest23190-21ssh10:15:043
1823199100,9cyclictest14826-21seq08:10:053
1822899109,1cyclictest30166-21sed08:40:192
1822899108,2cyclictest632-21sed08:45:242
1822899108,1cyclictest20410-21sed08:20:172
1822699108,1cyclictest7186-21chrt07:51:381
1822699108,1cyclictest2418-21chrt07:42:081
1822699108,1cyclictest22204-21ssh12:30:031
1822699108,1cyclictest16830-21chrt08:13:181
1822699107,2cyclictest9748-21chrt07:59:071
1822699107,2cyclictest29718-21sed08:40:161
1822699107,2cyclictest28670-21chrt08:37:301
1822699107,2cyclictest25499-21grep08:30:201
1822699101,1cyclictest0-21swapper/108:45:111
1822199108,2cyclictest24079-21cat07:20:170
1822199108,2cyclictest18496-21grep07:10:140
1822199108,1cyclictest38692sleep008:55:120
1822199108,1cyclictest119272sleep008:01:200
1822199107,2cyclictest3305-21expr07:45:140
1822199107,2cyclictest2396-21seq07:41:510
182319998,1cyclictest191722sleep308:17:453
182319997,1cyclictest24401-21aten2_r7power_c08:30:133
182319997,1cyclictest19881-21expr08:20:153
182319996,2cyclictest3826-21seq08:55:083
182289991,4cyclictest0-21swapper/208:25:132
182319986,1cyclictest7672-21which09:00:223
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional