You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 15:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sat Dec 06, 2025 12:44:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6924722171117,50sleep30-21swapper/307:07:103
69234428651,10sleep20-21swapper/207:05:332
68772127962,10sleep10-21swapper/107:05:041
69266027563,6sleep00-21swapper/007:09:360
11356062291,3sleep20-21swapper/212:29:262
9067442271,5sleep10-21swapper/110:05:131
692812992614,4cyclictest8675952sleep309:40:013
69281299261,3cyclictest0-21swapper/309:47:543
69281299261,2cyclictest0-21swapper/311:47:583
69281299261,2cyclictest0-21swapper/309:51:323
69281299260,2cyclictest0-21swapper/312:09:483
69281299260,2cyclictest0-21swapper/312:09:483
69281299251,2cyclictest0-21swapper/311:40:553
69281299251,2cyclictest0-21swapper/311:28:593
69281299251,19cyclictest1116904-21taskset12:16:513
69281299251,15cyclictest66550irq/131-enp1s0-TxRx-310:18:273
69281299250,2cyclictest0-21swapper/309:35:493
69281299241,3cyclictest0-21swapper/309:12:573
69281299241,11cyclictest0-21swapper/310:35:133
69281299240,2cyclictest0-21swapper/312:28:223
9752732231,8sleep1975282-21ssh10:50:111
9146962231,18sleep2914906-21hddtemp_smartct10:10:152
69281299235,15cyclictest15950irq/132-ahci[0000:00:17.0]08:36:473
69281299231,2cyclictest0-21swapper/309:59:223
9583562221,15sleep0958369-21sshd10:36:430
8503062221,3sleep1821854-21diskmemload09:26:201
8503062221,3sleep1821854-21diskmemload09:26:201
69281299229,8cyclictest898729-21sleep10:04:553
692812992219,1cyclictest0-21swapper/312:25:003
69281299221,16cyclictest816599-21sleep09:05:013
69281299221,11cyclictest0-21swapper/310:20:123
69281299221,10cyclictest920352-21taskset10:12:433
11510272221,3sleep00-21swapper/012:37:190
10771812221,4sleep00-21swapper/011:55:120
8990212211,3sleep20-21swapper/210:00:092
8899522211,2sleep20-21swapper/209:53:272
69281299219,3cyclictest0-21swapper/311:24:323
69281299212,14cyclictest996915-21taskset11:02:423
69281299212,14cyclictest988114-21taskset10:55:493
69281299212,13cyclictest951079-21taskset10:31:483
692812992118,1cyclictest0-21swapper/307:39:233
692812992115,3cyclictest0-21swapper/311:58:403
692812992111,8cyclictest0-21swapper/311:51:323
69281299211,16cyclictest800770-21turbostat08:50:003
69281299211,16cyclictest0-21swapper/312:30:143
69281299211,16cyclictest0-21swapper/312:00:163
69281299211,16cyclictest0-21swapper/310:45:113
69281299211,16cyclictest0-21swapper/309:10:003
69281299211,15cyclictest795268-21taskset08:41:363
69281299211,15cyclictest0-21swapper/311:15:123
69281299211,15cyclictest0-21swapper/311:15:123
69281299211,14cyclictest1041729-21taskset11:30:253
69281299211,13cyclictest937584-21taskset10:25:063
9881052201,3sleep20-21swapper/210:55:492
69281299205,12cyclictest774169-21turbostat08:29:553
69281299202,13cyclictest857093-21taskset09:30:343
69281299201,3cyclictest0-21swapper/311:13:013
69281299201,3cyclictest0-21swapper/309:28:223
69281299201,3cyclictest0-21swapper/309:28:223
692812992012,5cyclictest1006049-21turbostat11:10:003
69281299201,15cyclictest0-21swapper/312:10:113
69281299201,15cyclictest0-21swapper/309:15:133
69281299201,13cyclictest735818-21chrt07:45:173
69281299201,13cyclictest1152891-21taskset12:39:563
69281299200,1cyclictest0-21swapper/310:52:343
9826322191,5sleep0821854-21diskmemload10:54:510
9439292191,2sleep20-21swapper/210:27:592
9288972191,4sleep1821854-21diskmemload10:19:241
69281299191,15cyclictest0-21swapper/310:40:143
69281299191,15cyclictest0-21swapper/309:20:123
69281299191,14cyclictest0-21swapper/310:05:133
69281299191,12cyclictest705101-21taskset07:15:313
69281299191,12cyclictest1049240-21taskset11:35:263
69281299191,11cyclictest767568-21taskset08:15:193
11333612191,5sleep1821854-21diskmemload12:26:121
8739822181,3sleep10-21swapper/109:42:401
692812991815,1cyclictest0-21swapper/308:55:183
69281299181,4cyclictest742587-21sleep07:55:003
69281299180,11cyclictest692983-21taskset07:10:063
11195082182,10sleep11119534-21apt-get12:20:111
11008622181,2sleep00-21swapper/012:08:070
11008622181,2sleep00-21swapper/012:08:070
10192042180,7sleep20-21swapper/211:15:582
10192042180,7sleep20-21swapper/211:15:572
9881352171,3sleep10-21swapper/110:55:521
8993562171,7sleep0899410-21tune2fs10:00:150
8578192170,3sleep20-21swapper/209:31:502
8276192171,4sleep1821854-21diskmemload09:11:291
69281299171,12cyclictest753641-21chrt08:05:133
69281299171,12cyclictest710821-21sleep07:25:003
69281299171,11cyclictest779955-21chrt08:30:023
69281299171,11cyclictest718952-21taskset07:30:183
69281299170,13cyclictest0-21swapper/308:20:203
11189972171,2sleep1821854-21diskmemload12:19:521
10597902171,4sleep1821854-21diskmemload11:43:591
10501742171,4sleep2821854-21diskmemload11:36:342
100573021710,3sleep1821854-21diskmemload11:09:311
10043712171,3sleep20-21swapper/211:07:032
9970022161,10sleep0821854-21diskmemload11:02:530
9654062161,10sleep0965419-21rm10:40:440
8806022161,2sleep00-21swapper/009:45:390
7274292161,10sleep1727545-21irqstats07:40:151
69281299167,7cyclictest800770-21turbostat08:54:553
69281299161,9cyclictest742810-21taskset07:55:073
69281299161,12cyclictest763599-21sleep08:15:003
69281299160,13cyclictest0-21swapper/307:25:183
11509012161,9sleep1821854-21diskmemload12:37:071
11405162161,8sleep21140567-21ntpq12:30:222
10944462161,8sleep20-21swapper/212:05:172
10944462161,8sleep20-21swapper/212:05:172
10518572161,9sleep0821854-21diskmemload11:39:210
10371912161,4sleep1821854-21diskmemload11:30:041
9975322151,9sleep2821854-21diskmemload11:03:472
9801442150,1sleep20-21swapper/210:50:322
9579992151,3sleep1821854-21diskmemload10:36:041
9444252151,8sleep1821854-21diskmemload10:28:501
9442972150,1sleep00-21swapper/010:28:370
8908222153,2sleep0171rcu_preempt09:54:570
8426172150,1sleep00-21swapper/009:21:300
81685721510,2sleep00-21swapper/009:05:150
69281299151,9cyclictest731738-21taskset07:41:483
692812991513,1cyclictest0-21swapper/308:00:183
11045572151,9sleep11106206-21cut12:10:171
10359032150,1sleep10-21swapper/111:28:161
9667432140,3sleep1821854-21diskmemload10:43:171
9510862141,3sleep0821854-21diskmemload10:31:500
9368672141,8sleep1821854-21diskmemload10:24:371
9271882141,3sleep2821854-21diskmemload10:16:242
9210722141,8sleep0921083-21ssh10:14:100
8684402141,8sleep2868522-21cut09:40:162
8378612141,8sleep2838939-21kthreadcore09:20:162
7328202146,4sleep1734016-21cut07:45:161
11429812140,8sleep11142992-21ssh12:32:541
10213282141,4sleep01021338-21ssh11:19:440
10213282141,4sleep01021338-21ssh11:19:440
10147172142,6sleep11015979-21grep11:15:161
10147172142,6sleep11015979-21grep11:15:161
9270912130,4sleep00-21swapper/010:16:140
8446302131,8sleep1844663-21idleruntime-cro09:25:001
7805242131,7sleep2781252-21kthreadcore08:30:182
7265302131,7sleep2726599-31find07:39:242
11268012131,7sleep21126811-21ssh12:23:532
10855832132,5sleep01085612-21apt-get12:00:130
10843062131,3sleep21084320-21ssh11:58:472
10422952130,1sleep00-21swapper/011:30:470
9582352121,5sleep20-21swapper/210:36:292
9129082121,5sleep2821854-21diskmemload10:07:462
8805782121,4sleep1821854-21diskmemload09:45:361
8663772121,7sleep1866380-21ssh09:37:551
8522822122,5sleep1852300-21apt-get09:30:171
8513052121,4sleep0821854-21diskmemload09:29:000
8513052121,4sleep0821854-21diskmemload09:29:000
8498162120,1sleep20-21swapper/209:25:292
8498162120,1sleep20-21swapper/209:25:292
8015572121,7sleep0801627-21kthreadcore08:50:200
11176472120,1sleep20-21swapper/212:17:552
11119742121,7sleep00-21swapper/012:15:170
11089432121,7sleep01108956-21ssh12:12:360
11009692120,2sleep10-21swapper/112:08:161
11009692120,2sleep10-21swapper/112:08:161
10932302120,4sleep10-21swapper/112:04:191
10926162121,3sleep20-21swapper/212:03:282
10747872121,5sleep20-21swapper/211:52:142
10668572121,5sleep00-21swapper/011:47:180
10621802120,7sleep21063780-21kthreadcore11:45:202
10601752120,3sleep00-21swapper/011:44:380
10358352121,5sleep2821854-21diskmemload11:28:082
10268122121,3sleep20-21swapper/211:20:392
10120852121,5sleep1821854-21diskmemload11:12:041
9957962111,4sleep1821854-21diskmemload11:00:411
9830682111,4sleep00-21swapper/010:55:110
9746482111,5sleep1821854-21diskmemload10:49:021
9661082111,4sleep2821854-21diskmemload10:42:102
9520282111,4sleep1821854-21diskmemload10:34:151
9503792111,4sleep2821854-21diskmemload10:30:352
9366562111,4sleep0821854-21diskmemload10:24:120
9350902111,4sleep2821854-21diskmemload10:21:252
9204042111,4sleep1821854-21diskmemload10:12:501
9127332111,5sleep00-21swapper/010:07:280
8975502111,4sleep1821854-21diskmemload09:58:031
8953202115,1sleep20-21swapper/209:55:202
8911842111,3sleep00-21swapper/009:55:100
8601812111,4sleep2821854-21diskmemload09:35:072
8595702111,3sleep00-21swapper/009:34:570
8363672111,6sleep20-21swapper/209:19:042
8353832111,4sleep1821854-21diskmemload09:17:181
7802522111,5sleep0780354-21perl08:30:160
7709302111,6sleep0772743-21kthreadcore08:20:180
7696242111,5sleep2769695-21irqstats08:20:162
7644032110,7sleep1764435-21kthreadcore08:15:181
7586242111,6sleep0758743-21cpuspeed_turbos08:10:120
7428182111,4sleep1742837-21apt-get07:55:131
7380802111,6sleep1738824-21kthreadcore07:50:171
7266432111,5sleep1726599-31find07:39:311
7110802111,5sleep2711338-21df_inode07:25:132
7098882111,6sleep0709933-21/usr/sbin/munin07:20:220
11502022111,3sleep20-21swapper/212:36:072
11437212111,4sleep0821854-21diskmemload12:33:570
11333482111,3sleep00-21swapper/012:26:110
11076852111,4sleep2821854-21diskmemload12:10:502
10779202111,6sleep11079298-21kthreadcore11:55:171
10765102111,4sleep0821854-21diskmemload11:54:560
10678242111,4sleep1821854-21diskmemload11:48:561
10588552111,3sleep20-21swapper/211:42:312
10506772111,4sleep1821854-21diskmemload11:37:251
10425752111,3sleep20-21swapper/211:31:172
10268872111,4sleep1821854-21diskmemload11:20:461
10267672111,3sleep00-21swapper/011:20:330
10125312111,2sleep00-21swapper/011:12:510
10049242111,4sleep0821854-21diskmemload11:08:020
9731162101,2sleep20-21swapper/210:46:192
9678352101,4sleep0968102-21df_inode10:45:130
8994132101,3sleep1821854-21diskmemload10:00:161
8884222101,4sleep1821854-21diskmemload09:50:461
8818812101,5sleep2821854-21diskmemload09:47:542
8721042101,3sleep00-21swapper/009:40:220
8606752101,3sleep00-21swapper/009:35:170
8298862101,3sleep00-21swapper/009:15:050
8284222101,2sleep00-21swapper/009:12:560
8276652101,3sleep20-21swapper/209:11:342
8207152101,4sleep2820836-21ntpq09:05:212
8007582100,6sleep1800777-21latency_hist08:50:011
7997832101,4sleep0799830-21ntpq08:45:230
7323212101,4sleep2732537-21diskstats07:45:132
7318982100,4sleep00-21swapper/007:43:510
7275462101,4sleep2728644-21kthreadcore07:40:162
7266422100,4sleep0726552-31sort07:39:300
7057392101,4sleep2705758-21apt-get07:20:122
6931582101,4sleep2693281-21diskstats07:10:142
11262562101,3sleep0821854-21diskmemload12:23:070
10695192100,2sleep1171rcu_preempt11:50:161
10350622101,3sleep00-21swapper/011:26:500
10113422101,3sleep20-21swapper/211:10:442
816572291,3sleep2816596-21munin-run09:05:012
806293291,2sleep10-21swapper/108:55:141
774421291,3sleep2774440-21apt-get08:25:132
741593291,4sleep2741647-21ldconfig07:50:222
716330291,3sleep10-21swapper/107:30:131
715064294,1sleep10-21swapper/107:25:201
693565291,4sleep1695020-21grep07:10:171
812613281,3sleep1814326-21cut09:00:191
773537280,1sleep10-21swapper/108:20:271
768771280,1sleep20-21swapper/208:19:082
763368283,2sleep2802-21snmpd08:12:302
758343281,3sleep0567-21dbus-daemon08:10:000
747701280,1sleep20-21swapper/207:58:272
741426281,3sleep0741539-21ntpq07:50:200
717258281,3sleep2718951-21cut07:30:182
716060280,1sleep00-21swapper/007:29:560
705456281,3sleep2705482-21munin-run07:20:002
697011280,2sleep00-21swapper/007:10:200
811043270,1sleep00-21swapper/008:57:010
801298270,1sleep20-21swapper/208:50:182
790226270,2sleep20-21swapper/208:39:322
785816271,2sleep00-21swapper/008:35:170
779422270,4sleep00-21swapper/008:29:560
768498270,1sleep00-21swapper/008:15:410
753066270,1sleep10-21swapper/108:04:401
751992270,3sleep20-21swapper/208:00:202
811563261,1sleep00-21swapper/009:00:140
801895261,2sleep10-21swapper/108:50:211
794847261,2sleep1790551-21/usr/sbin/munin08:40:271
794561260,2sleep00-21swapper/008:40:220
790550260,2sleep20-21swapper/208:40:122
789927260,4sleep10-21swapper/108:35:421
784794260,1sleep10-21swapper/108:32:451
779224260,1sleep10-21swapper/108:27:241
762874261,1sleep10-21swapper/108:10:251
753924261,2sleep10-21swapper/108:05:161
749440261,2sleep00-21swapper/008:00:180
747497260,4sleep00-21swapper/007:55:490
716658260,2sleep00-21swapper/007:30:160
705400260,1sleep10-21swapper/107:19:191
698149261,2sleep00-21swapper/007:15:010
817117250,2sleep10-21swapper/109:05:161
807763250,2sleep20-21swapper/208:55:192
754984250,1sleep20-21swapper/208:05:172
732749250,1sleep00-21swapper/007:45:150
796095240,1sleep20-21swapper/208:45:172
705727240,1sleep10-21swapper/107:20:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional