You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-08 - 23:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Thu Jan 08, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2008829267,9sleep20-21swapper/207:06:362
2024528465,15sleep00-21swapper/007:08:380
2022828269,9sleep30-21swapper/307:08:263
2007728164,13sleep10-21swapper/107:06:261
2044499287,21cyclictest0-21swapper/211:40:492
2044499230,0cyclictest0-21swapper/207:52:012
2043999220,16cyclictest0-21swapper/009:15:120
2043999220,16cyclictest0-21swapper/009:15:120
20448992018,1cyclictest28275-21munin-run10:55:013
20448991914,3cyclictest39-21ksoftirqd/309:25:003
2044499191,15cyclictest1962-21apt-get08:50:122
2044499190,18cyclictest0-21swapper/212:26:242
20448991713,1cyclictest39-21ksoftirqd/312:05:143
20448991710,6cyclictest39-21ksoftirqd/308:50:133
2043999174,6cyclictest14794-21apt-get08:09:590
20448991614,1cyclictest29634-21apt-get08:40:123
2043999166,3cyclictest273-21systemd-udevd11:07:040
2044899159,2cyclictest7594-21cron12:30:003
2044899154,11cyclictest39-21ksoftirqd/309:30:133
20448991513,1cyclictest23356-21munin-run11:55:003
2044499155,10cyclictest0-21swapper/211:02:412
2044199150,15cyclictest0-21swapper/110:50:251
20448991411,2cyclictest27001-21wc12:00:183
20448991410,3cyclictest20373-21wc11:45:253
20448991410,3cyclictest14293-21tr09:15:143
20448991410,3cyclictest14293-21tr09:15:133
2044899140,13cyclictest1083-21wc08:45:253
2044899140,12cyclictest0-21swapper/309:40:143
20444991414,0cyclictest0-21swapper/211:45:132
20444991411,2cyclictest1799-21sh10:00:002
2044199149,4cyclictest32707-21tr07:35:131
20441991410,3cyclictest26253-21grep12:00:131
2043999149,3cyclictest28627-21cron09:45:000
2043999145,8cyclictest16313-21munin-run10:30:010
2043999145,5cyclictest225-21systemd-journal09:00:010
2043999143,6cyclictest28147-21idleruntime-cro12:05:000
20439991410,2cyclictest29216-21grep09:45:130
2044899139,3cyclictest4185-21tail07:40:253
2044899139,3cyclictest28438-21cut07:25:163
2044899139,3cyclictest26170-21cat09:40:003
2044899139,3cyclictest25083-21tr08:30:153
2044899139,3cyclictest19988-21wc10:35:193
2044899138,1cyclictest25888-21ls10:50:003
2044899132,5cyclictest0-21swapper/308:35:123
2044899131,9cyclictest39-21ksoftirqd/311:30:253
20448991310,2cyclictest20940-21tr07:10:133
20448991310,2cyclictest15973-21grep08:10:143
20448991310,2cyclictest12240-21cut09:10:163
20444991313,0cyclictest0-21swapper/212:10:202
20444991310,2cyclictest8827-21cron09:05:002
20444991310,2cyclictest32756-21sed08:45:152
20444991310,2cyclictest32194-21grep09:50:162
20444991310,2cyclictest29087-21cat11:00:002
20444991310,2cyclictest27581-21cat09:40:202
20444991310,2cyclictest24944-21cat09:35:192
2044199139,3cyclictest30752-21tr08:40:201
2044199139,2cyclictest10263-21grep07:55:151
2044199131,2cyclictest141rcu_preempt08:50:191
20441991310,2cyclictest7650-21grep11:20:161
20441991310,2cyclictest7373-21grep09:00:161
20441991310,2cyclictest533-21ls11:05:231
20441991310,2cyclictest31498-21tail12:10:171
20441991310,2cyclictest28874-21sort09:45:001
20441991310,2cyclictest24663-21grep07:15:261
20441991310,2cyclictest15673-21sed11:35:261
20441991310,2cyclictest12267-21sed09:10:161
2043999139,3cyclictest7104-21grep12:25:260
2043999139,3cyclictest5221-21cron12:25:000
2043999139,3cyclictest25165-21sleep07:20:000
2043999139,2cyclictest25972-21cut08:30:210
2043999139,2cyclictest17191-21sed11:40:160
2043999136,6cyclictest7037-21munin-run07:50:000
2043999131,6cyclictest1826-21tr09:59:590
20439991310,2cyclictest962-21tr12:15:130
20439991310,2cyclictest5004-21grep11:15:150
20439991310,2cyclictest31140-21wc08:40:260
2043999130,11cyclictest2223-21tr08:50:130
2044899129,2cyclictest9898-21grep07:55:133
2044899129,2cyclictest7369-21cut09:00:163
2044899129,2cyclictest6651-21rm11:20:013
2044899129,2cyclictest6014-21grep07:45:203
2044899129,2cyclictest32115-21cut12:10:223
2044899129,2cyclictest2728-21sort10:00:123
2044899129,2cyclictest25935-21df07:20:153
2044899129,2cyclictest23008-21sed10:40:243
2044899129,2cyclictest21802-21grep08:20:253
2044899129,2cyclictest18391-21grep08:15:173
2044899129,2cyclictest15643-21grep11:35:263
2044899129,2cyclictest12106-21grep10:20:133
2044899128,3cyclictest10891-21uniq12:35:173
2044899127,4cyclictest30672-21ls11:00:213
2044899120,11cyclictest16706-21tr11:40:133
2044499129,2cyclictest5681-21sed07:45:172
2044499129,2cyclictest32302-21apt07:35:002
2044499129,2cyclictest2797-21grep07:40:132
2044499129,2cyclictest27634-21grep12:00:252
2044499129,2cyclictest21130-21sleep10:40:002
2044499129,2cyclictest13271-21grep11:30:242
2044499129,2cyclictest12342-21tr08:00:172
2044499129,2cyclictest10396-21ls10:15:182
2044499128,3cyclictest0-21swapper/207:25:212
2044499124,8cyclictest0-21swapper/211:11:412
2044499124,7cyclictest25928-21df07:20:162
2044499121,1cyclictest21548-21apt-get09:30:142
2044499121,10cyclictest20666-21apache_processe07:10:112
2044499121,10cyclictest14794-21apt-get08:10:012
2044499120,11cyclictest23187-21cut07:15:142
2044199129,2cyclictest5514-21cat10:05:191
2044199129,2cyclictest27960-21tr07:25:141
2044199129,2cyclictest26959-21cron08:35:001
2044199129,2cyclictest22430-21cat10:40:181
2044199129,2cyclictest21139-21cat08:20:181
2044199129,2cyclictest19761-21sed10:35:161
2044199129,2cyclictest18744-21cat10:35:001
2044199129,2cyclictest18350-21grep09:20:261
2044199129,2cyclictest1282-21cat09:54:591
2044199128,3cyclictest6935-21cat12:25:251
2044199127,3cyclictest0-21swapper/112:15:141
2044199121,7cyclictest225-21systemd-journal07:50:001
2044199121,10cyclictest0-21swapper/108:15:141
20441991210,1cyclictest16303-21ls10:30:011
20441991210,1cyclictest13054-21tr10:20:211
20441991210,1cyclictest12371-21grep11:30:151
2044199120,1cyclictest141rcu_preempt11:50:241
2044199120,11cyclictest26309-21cat09:40:001
2044199120,11cyclictest14821-21head09:15:161
2044199120,11cyclictest14821-21head09:15:151
2043999129,2cyclictest8955-21cut12:30:180
2043999129,2cyclictest8539-21sed11:20:260
2043999129,2cyclictest804-21cut07:35:160
2043999129,2cyclictest7432-21grep09:00:160
2043999129,2cyclictest6113-21grep10:05:250
2043999129,2cyclictest32247-21wc12:10:240
2043999129,2cyclictest29110-21head12:05:150
2043999129,2cyclictest2855-21cut10:00:120
2043999129,2cyclictest27254-21head10:50:180
2043999129,2cyclictest19735-21grep10:35:160
2043999129,2cyclictest19266-21cut09:25:130
2043999129,2cyclictest19151-21wc08:15:260
2043999129,2cyclictest17378-21sh08:14:590
2043999129,2cyclictest13263-21wc10:20:240
2043999128,3cyclictest12748-21cut11:30:170
2043999128,2cyclictest7521-21tr07:50:140
2043999128,2cyclictest24603-21dump-pmu-power08:29:590
2043999127,4cyclictest14171-21cron08:04:590
2043999120,11cyclictest22763-21head09:30:220
2044899119,1cyclictest5230-21cut10:05:173
2044899118,2cyclictest7522-21tr07:50:133
2044899118,2cyclictest672-21grep09:50:253
2044899118,2cyclictest32002-21cut11:05:153
2044899118,2cyclictest25505-21chrt11:57:183
2044899118,2cyclictest2438-21tail11:10:153
2044899118,2cyclictest21297-21basename09:30:003
2044899118,2cyclictest1656-21wc07:35:243
2044899118,2cyclictest1499-21cut12:15:163
2044899117,3cyclictest12294-21tr08:00:173
20448991110,1cyclictest381rcuc/311:20:153
2044499119,1cyclictest16465-21systemctl09:20:012
2044499119,1cyclictest16465-21systemctl09:20:002
2044499119,1cyclictest10993-21grep11:25:252
2044499119,1cyclictest10516-21grep12:35:142
2044499118,2cyclictest8823-21cat12:30:172
2044499118,2cyclictest7296-21cat11:20:142
2044499118,2cyclictest5811-21cat08:55:272
2044499118,2cyclictest4811-21sed12:20:242
2044499118,2cyclictest22051-21cut10:40:152
2044499118,2cyclictest20395-21tr08:20:132
2044499118,2cyclictest18828-21cron09:25:002
2044499118,2cyclictest17289-21grep10:30:152
2044499118,2cyclictest14792-21head11:35:172
2044499118,2cyclictest1340-21ls07:35:202
2044499118,2cyclictest12190-21grep09:10:162
2044499111,9cyclictest29309-21irqstats12:05:172
2044499111,9cyclictest23616-21apache_processe11:55:112
2044499111,5cyclictest225-21systemd-journal11:20:002
2044499110,2cyclictest19980-21basename09:25:172
2044499110,10cyclictest0-21swapper/210:47:472
2044499110,10cyclictest0-21swapper/208:15:132
2044199119,1cyclictest22973-21df08:25:141
2044199119,1cyclictest20867-21head09:25:251
2044199118,2cyclictest8413-21grep12:30:141
2044199118,2cyclictest4735-21grep12:20:241
2044199118,2cyclictest31282-21munin-plugin-st11:05:001
2044199118,2cyclictest30762-21sed09:45:241
2044199118,2cyclictest29609-21head12:05:201
2044199118,2cyclictest26577-21ntpq07:20:211
2044199118,2cyclictest13377-21gzip08:00:251
2044199111,9cyclictest38150irq/131-snd_hda07:50:121
2044199110,1cyclictest141rcu_preempt08:35:251
2043999118,2cyclictest7542-21tail10:10:180
2043999118,2cyclictest4116-21wc07:40:250
2043999118,2cyclictest30301-21grep07:30:140
2043999118,2cyclictest29267-21cut11:00:090
2043999118,2cyclictest28548-21head07:25:170
2043999118,2cyclictest25555-21sed09:35:250
2043999118,2cyclictest21277-21wget11:50:120
2043999118,2cyclictest20806-21grep07:10:130
2043999118,2cyclictest20325-21grep08:20:130
2043999118,2cyclictest17670-21head10:30:180
2043999118,2cyclictest1334-21chrt08:45:340
2043999118,2cyclictest11768-21cron08:00:000
2043999118,2cyclictest11043-21chrt10:16:370
2043999117,3cyclictest1164-21cron09:54:590
2043999114,6cyclictest25277-21unixbench_multi11:55:250
2044899108,1cyclictest7983-21cut12:30:133
2044899108,1cyclictest320202sleep307:31:293
2044899108,1cyclictest286462sleep310:57:073
2044899108,1cyclictest23180-21cut08:25:173
2044899108,1cyclictest18550-21seq10:33:323
2044899108,1cyclictest160492sleep310:26:543
2044899107,2cyclictest23565-21sed07:15:153
2044899107,2cyclictest11265-21chrt10:19:293
2044899106,3cyclictest5436-21grep08:55:203
2044899106,3cyclictest30773-21grep09:45:243
2044899106,2cyclictest11217-21sendmail09:10:013
2044899105,4cyclictest381rcuc/310:10:263
2044899103,6cyclictest46-21khungtaskd11:25:133
2044499108,1cyclictest4562-21wget10:05:142
2044499108,1cyclictest31907-21cut08:45:012
2044499108,1cyclictest26442-21cstates10:50:142
2044499108,1cyclictest24824-21taskset08:30:072
2044499107,2cyclictest8605-21taskset10:10:502
2044499107,2cyclictest856-21egrep12:15:132
2044499107,2cyclictest32-21ksoftirqd/210:20:202
2044499107,2cyclictest28827-21cat08:35:252
2044499107,2cyclictest22527-21chrt08:25:142
2044499104,2cyclictest225-21systemd-journal10:04:592
2044499102,4cyclictest225-21systemd-journal09:05:012
2044499102,3cyclictest0-21swapper/210:25:142
2044499101,8cyclictest9660-21apache_accesses07:55:102
2044199108,1cyclictest68422sleep110:10:141
2044199108,1cyclictest14621-21taskset08:07:561
2044199107,2cyclictest4608-21grep08:55:161
2044199107,2cyclictest32241-21tr08:45:131
2044199107,2cyclictest31254-21wc07:30:201
2044199107,2cyclictest20724-21aten2_r7power_e07:10:131
2044199107,2cyclictest15145-21rm08:10:001
2044199107,2cyclictest11317-21cat09:10:001
2044199107,2cyclictest10929-21grep11:25:251
2044199107,2cyclictest10521-21fgrep10:15:181
2044199103,3cyclictest225-21systemd-journal11:20:001
2043999108,1cyclictest20843-21taskset11:48:250
2043999108,1cyclictest1710-21apache_processe11:10:100
2043999107,2cyclictest9998-21grep11:25:160
2043999107,2cyclictest9744-21cut09:05:150
2043999107,2cyclictest28791-21taskset10:59:000
2043999107,2cyclictest25491-21basename07:20:140
2043999107,2cyclictest24417-21cut10:45:130
2043999107,2cyclictest23257-21sshd10:41:540
2043999107,2cyclictest10201-21chrt12:35:050
2043999106,3cyclictest12719-21head09:10:230
2043999105,3cyclictest27301-21grep08:35:120
2043999102,4cyclictest225-21systemd-journal11:40:000
204489997,1cyclictest146492sleep308:08:193
204489996,2cyclictest39-21ksoftirqd/310:00:003
204489990,2cyclictest5238-21grep12:25:003
204449997,2cyclictest21521-21tr11:50:122
204449996,2cyclictest32024-21tr11:05:152
204449990,1cyclictest0-21swapper/208:07:392
204419998,1cyclictest2932-21mii-tool10:00:131
204419998,1cyclictest23310-21taskset09:30:471
204419997,2cyclictest24202-21cat10:45:121
204419997,2cyclictest12029-21taskset12:36:021
204419997,1cyclictest28600-21taskset10:56:321
204419997,1cyclictest2508-21taskset07:40:061
204419997,1cyclictest23748-21hostname11:55:131
204419997,1cyclictest16616-21grep11:40:131
204419990,9cyclictest2633-21kernelversion11:10:171
204419990,9cyclictest20179-21find11:45:221
204399996,2cyclictest16969-21tr09:20:130
204449986,2cyclictest29025-21cat09:45:012
204419986,2cyclictest1728-21taskset09:59:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional