You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-31 - 14:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Aug 31, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1627529669,9sleep20-21swapper/207:08:252
1610029379,9sleep00-21swapper/007:06:090
1606629142,45sleep10-21swapper/107:05:431
1637828269,9sleep30-21swapper/307:09:473
62032500,0sleep00-21swapper/009:26:490
321012460,0sleep10-21swapper/110:59:511
174282310,0chrt17429-21sh10:31:010
1649199310,1cyclictest0-21swapper/312:11:043
1649199302,27cyclictest0-21swapper/311:46:173
1649199300,28cyclictest0-21swapper/312:00:473
1649199296,19cyclictest225-21systemd-journal10:30:003
1649199291,27cyclictest11264-21grep09:50:013
1649199290,1cyclictest0-21swapper/309:15:133
1649199282,25cyclictest9291-21diskmemload11:18:233
1649199280,27cyclictest0-21swapper/309:55:243
1649199280,1cyclictest0-21swapper/309:44:453
1649199272,24cyclictest9291-21diskmemload12:06:513
1649199272,24cyclictest0-21swapper/309:40:013
1649199271,1cyclictest9291-21diskmemload11:58:053
1649199270,26cyclictest0-21swapper/309:30:113
1649199270,26cyclictest0-21swapper/309:30:113
1649199270,25cyclictest9291-21diskmemload12:24:063
1649199270,1cyclictest0-21swapper/311:31:223
1649199264,21cyclictest9291-21diskmemload11:41:103
1649199261,24cyclictest0-21swapper/311:10:103
1649199261,22cyclictest0-21swapper/311:21:433
18962250,0sleep30-21swapper/310:21:583
1649199251,21cyclictest9291-21diskmemload11:07:223
1649199251,21cyclictest9291-21diskmemload11:07:213
1649199250,23cyclictest0-21swapper/309:50:513
1649199250,1cyclictest0-21swapper/311:25:143
1648899250,24cyclictest0-21swapper/210:55:002
1648499251,1cyclictest0-21swapper/110:06:351
1649199242,20cyclictest9291-21diskmemload09:14:173
1649199241,23cyclictest0-21swapper/311:50:183
1649199240,23cyclictest0-21swapper/310:50:023
1649199240,23cyclictest0-21swapper/310:14:573
16488992423,1cyclictest0-21swapper/210:37:052
1649199232,20cyclictest9291-21diskmemload10:07:553
1649199230,2cyclictest0-21swapper/311:01:173
1649199230,2cyclictest0-21swapper/310:31:263
1648899230,23cyclictest0-21swapper/211:53:062
1649199227,14cyclictest27727-21sh10:56:263
1649199225,16cyclictest9291-21diskmemload12:29:463
1649199221,2cyclictest9291-21diskmemload10:01:063
1649199220,3cyclictest5400-21ssh09:26:143
1649199220,21cyclictest0-21swapper/312:35:193
1649199220,21cyclictest0-21swapper/310:45:033
1649199220,1cyclictest0-21swapper/310:39:203
1649199220,1cyclictest0-21swapper/307:15:143
1649199220,19cyclictest24132-21wc08:30:233
1649199210,3cyclictest27689-21awk09:20:173
1649199210,3cyclictest0-21swapper/312:32:203
1649199210,1cyclictest0-21swapper/311:38:503
1649199210,19cyclictest0-21swapper/312:15:563
1648899210,21cyclictest0-21swapper/210:02:382
1648899210,19cyclictest0-21swapper/211:29:272
1648499210,20cyclictest19580-21processes09:15:211
1649199202,16cyclictest1087-21dpkg07:45:003
1649199200,3cyclictest25478-21grep07:25:263
1649199200,1cyclictest15693-21acpi08:15:113
1649199200,1cyclictest0-21swapper/307:45:133
1649199200,19cyclictest9291-21diskmemload10:43:063
1648899200,20cyclictest0-21swapper/211:19:072
16484992017,2cyclictest0-21swapper/111:56:471
1648499200,19cyclictest0-21swapper/110:12:101
16482992020,0cyclictest0-21swapper/009:35:550
1648899194,10cyclictest0-21swapper/207:30:142
1648899193,10cyclictest0-21swapper/209:13:422
1648899191,1cyclictest0-21swapper/208:50:132
1648499190,1cyclictest0-21swapper/109:14:471
1648499190,19cyclictest0-21swapper/109:38:151
1648899180,18cyclictest0-21swapper/210:42:492
1648899180,16cyclictest0-21swapper/211:59:462
16484991813,3cyclictest0-21swapper/112:30:321
1648499180,17cyclictest9291-21diskmemload11:25:071
1649199173,14cyclictest9291-21diskmemload10:15:203
16488991714,2cyclictest2909-21latency_hist09:45:002
1648899170,7cyclictest0-21swapper/209:55:362
1648899170,17cyclictest0-21swapper/212:03:222
1648899170,17cyclictest0-21swapper/209:21:112
1648499170,0cyclictest0-21swapper/110:47:481
1648299176,6cyclictest18780-21awk11:50:000
1649199165,10cyclictest0-21swapper/307:30:213
1649199160,1cyclictest0-21swapper/308:55:223
1648899169,1cyclictest0-21swapper/210:51:572
16488991616,0cyclictest0-21swapper/209:46:462
1648899160,2cyclictest0-21swapper/211:45:402
1648899160,16cyclictest0-21swapper/211:36:562
1648899160,16cyclictest0-21swapper/211:01:262
1648499167,8cyclictest5890-21apache212:37:001
16484991610,4cyclictest537-21snmpd11:01:161
1648499160,1cyclictest0-21swapper/109:49:051
1648299162,6cyclictest21251-21sendmail-msp07:20:000
16482991615,0cyclictest0-21swapper/012:20:060
1649199159,5cyclictest27713-21grep08:40:143
1649199158,6cyclictest6225-21cut07:55:153
1649199158,6cyclictest4798-21expr07:50:203
1649199158,6cyclictest21761-21expr07:20:143
1649199158,6cyclictest19457-21expr08:20:233
1649199158,6cyclictest13066-21seq08:09:573
1649199157,7cyclictest8739-21egrep08:00:143
1649199157,7cyclictest1021-21cat08:50:193
1649199151,13cyclictest25207-21taskset08:35:023
1649199150,1cyclictest0-21swapper/308:50:003
16488991514,0cyclictest0-21swapper/212:06:022
16488991513,1cyclictest32-21ksoftirqd/212:29:572
16488991513,1cyclictest32-21ksoftirqd/212:12:032
1648899151,1cyclictest9291-21diskmemload10:22:482
1648899151,0cyclictest141rcu_preempt12:15:532
1648899150,15cyclictest0-21swapper/210:46:042
1648499150,1cyclictest0-21swapper/110:32:321
1648499150,12cyclictest29746-21cron08:44:591
1648499150,12cyclictest0-21swapper/112:10:521
1648299153,6cyclictest22485-21ssh11:13:000
1648299153,6cyclictest0-21swapper/008:25:120
16482991514,1cyclictest13-21ksoftirqd/012:27:330
1649199146,7cyclictest18525-21taskset07:11:183
1649199140,4cyclictest225-21systemd-journal08:12:413
1649199140,1cyclictest0-21swapper/309:00:253
1649199140,1cyclictest0-21swapper/308:25:143
1649199140,14cyclictest0-21swapper/309:05:163
16488991413,1cyclictest311rcuc/208:05:102
16488991411,2cyclictest5587-21chrt09:26:232
16488991411,2cyclictest30950-21grep07:40:002
16488991411,2cyclictest30950-21grep07:40:002
16488991410,3cyclictest3114-21sed08:55:182
16488991410,3cyclictest18470-21ssh11:10:182
16488991410,2cyclictest6022-21grep12:20:212
1648899140,1cyclictest0-21swapper/212:35:142
1648899140,14cyclictest0-21swapper/211:08:542
1648899140,14cyclictest0-21swapper/211:08:532
1648499141,12cyclictest26223-21diskstats11:15:161
16484991410,2cyclictest22976-21grep08:30:141
1648499140,4cyclictest225-21systemd-journal12:25:001
1648499140,2cyclictest0-21swapper/110:20:181
1648299141,7cyclictest11251-21apt-get09:50:000
1648299140,1cyclictest0-21swapper/012:16:560
1649199136,6cyclictest29157-21expr07:35:163
1649199136,6cyclictest29157-21expr07:35:153
1648899139,4cyclictest291irq_work/210:25:062
1648899135,6cyclictest0-21swapper/211:40:142
1648899132,10cyclictest1142-21runrttasks11:24:552
1648899131,6cyclictest32-21ksoftirqd/210:20:002
16488991310,2cyclictest7322-21tr07:55:212
16488991310,2cyclictest32-21ksoftirqd/208:19:592
16488991310,2cyclictest30089-21grep08:45:142
1648899130,8cyclictest0-21swapper/210:32:162
1648899130,13cyclictest0-21swapper/211:31:232
1648499139,3cyclictest434-21grep07:40:261
1648499139,3cyclictest4330-21ssh12:00:171
1648499139,3cyclictest3570-21grep08:55:231
1648499139,3cyclictest10696-21cron08:05:001
1648499139,2cyclictest9411-21apt11:25:011
1648499139,2cyclictest26993-21cut07:30:181
1648499139,2cyclictest2144-21tail07:45:181
1648499138,4cyclictest22781-21ssh09:55:371
1648499137,5cyclictest25-21ksoftirqd/110:27:101
1648499135,2cyclictest225-21systemd-journal08:14:161
16484991310,2cyclictest23955-21head10:15:251
16484991310,2cyclictest18869-21cat11:49:591
1648499130,2cyclictest0-21swapper/109:54:041
1648499130,12cyclictest4801-21sort07:50:201
1648299139,3cyclictest1755-21grep11:20:140
1648299137,5cyclictest14203-21perf10:30:010
1648299135,6cyclictest5890-21apache212:00:050
1648299132,6cyclictest31062-21munin-run10:40:000
16482991310,2cyclictest6950-21wc10:05:240
16482991310,2cyclictest5922-21cron07:55:000
16482991310,2cyclictest30450-21grep07:35:270
16482991310,2cyclictest30450-21grep07:35:260
16482991310,2cyclictest18515-21expr09:15:140
16482991310,2cyclictest10630-21head09:10:180
16482991310,1cyclictest9291-21diskmemload11:05:070
16482991310,1cyclictest9291-21diskmemload11:05:070
1648299130,13cyclictest0-21swapper/010:22:150
1648899129,2cyclictest7892-21grep09:05:172
1648899129,2cyclictest29249-21sort12:35:012
1648899129,2cyclictest23203-21grep07:20:272
1648899129,2cyclictest20040-21sed09:15:252
1648899129,2cyclictest18382-21tr08:20:152
1648899129,2cyclictest12208-21grep09:30:202
1648899129,2cyclictest12208-21grep09:30:202
1648899128,3cyclictest29127-21cat08:40:262
1648899128,3cyclictest20299-21wc09:35:212
1648899121,2cyclictest141rcu_preempt10:05:192
16488991210,1cyclictest25966-21grep08:35:162
1648499129,2cyclictest698-21head08:50:161
1648499129,2cyclictest28046-21df09:40:141
1648499129,2cyclictest24068-21tail10:35:181
1648499129,2cyclictest12083-21basename12:05:151
1648499129,2cyclictest11751-21sed08:05:161
1648499121,10cyclictest9291-21diskmemload11:51:431
1648499120,1cyclictest0-21swapper/111:40:111
1648499120,1cyclictest0-21swapper/111:06:071
1648499120,1cyclictest0-21swapper/111:06:071
1648499120,11cyclictest13865-21grep12:25:181
1648299129,2cyclictest8335-21readlink08:00:000
1648299129,2cyclictest32373-21grep10:40:170
1648299129,2cyclictest30491-21grep12:35:160
1648299129,2cyclictest27896-21tr11:55:150
1648299129,2cyclictest27172-21grep09:20:150
1648299129,2cyclictest21662-21expr12:30:140
1648299129,2cyclictest20954-21wc11:50:260
1648299129,2cyclictest1994-21sort07:45:170
1648299129,2cyclictest14209-21sed09:50:400
1648299129,2cyclictest1274-21grep11:00:180
1648299129,2cyclictest10299-21head08:00:270
1648299128,3cyclictest28502-21ssh10:00:000
1648299128,3cyclictest28273-21head09:40:160
1648299128,3cyclictest16442-21grep08:15:170
1648299121,7cyclictest28561-21apt-config07:35:010
1648299121,6cyclictest0-21swapper/008:55:140
16482991211,1cyclictest0-21swapper/011:15:080
1648299121,10cyclictest23639-21head08:30:170
1648299120,10cyclictest8336-21head10:45:170
1648899119,1cyclictest3009-21sed07:45:272
1648899118,2cyclictest6984-21cat09:05:002
1648899118,2cyclictest385-21grep07:40:262
1648899118,2cyclictest24981-21uname07:25:202
1648899118,2cyclictest17395-21ssh10:11:472
1648899115,2cyclictest225-21systemd-journal08:29:212
1648899110,10cyclictest11918-21ssh09:50:112
1648499119,1cyclictest16107-21chrt10:50:131
1648499118,2cyclictest7192-21cat07:55:191
1648499118,2cyclictest3885-21systemctl09:25:211
1648499118,2cyclictest31075-21ssh12:15:471
1648499118,2cyclictest29950-21tail10:00:161
1648499118,2cyclictest26640-21sh11:35:131
1648499118,2cyclictest24091-21tr07:25:151
1648499118,2cyclictest19882-21tail07:15:171
1648499118,2cyclictest19656-21sed11:30:231
1648499118,2cyclictest1864-21ssh10:41:001
1648499118,2cyclictest17937-21tr11:10:141
1648499118,2cyclictest16777-21cut07:10:121
1648499118,2cyclictest15707-21ssh09:32:461
1648499118,2cyclictest15707-21ssh09:32:451
1648499116,4cyclictest27321-21cat09:20:151
1648299119,1cyclictest18398-21grep08:20:140
1648299118,2cyclictest3234-21taskset11:40:160
1648299118,2cyclictest30781-21sed08:45:180
1648299118,2cyclictest27786-21ssh10:56:290
1648299118,2cyclictest23247-21tr10:15:200
1648299118,2cyclictest22051-21ssh11:31:490
1648299118,2cyclictest21590-21sh12:10:210
1648299118,2cyclictest18589-21ssh10:12:440
1648299118,2cyclictest18512-21ssh12:09:010
1648299118,2cyclictest18051-21sed10:50:280
1648299118,2cyclictest17417-21grep07:10:160
1648299118,2cyclictest14484-21ssh09:31:480
1648299118,2cyclictest14484-21ssh09:31:470
1648299118,2cyclictest13580-21ssh11:26:440
1648299118,2cyclictest12637-21grep08:05:250
1648299117,3cyclictest29396-21grep10:00:130
1648299112,7cyclictest32392-21apache_volume08:50:120
1648299110,10cyclictest27650-21dpkg08:40:130
1648299110,10cyclictest0-21swapper/011:37:270
1648899108,1cyclictest24698-21taskset08:31:272
1648899108,1cyclictest20566-21sed07:15:262
1648899107,2cyclictest5139-21tr07:50:242
1648899107,2cyclictest18380-21tail07:10:252
1648499108,1cyclictest31986-21sshd08:48:221
1648499108,1cyclictest26366-21ntp_kernel_err08:35:191
1648499107,2cyclictest7720-21sed09:05:141
1648499107,2cyclictest4772-21cut09:00:141
1648499107,2cyclictest30766-21chrt07:37:571
1648499107,2cyclictest30766-21chrt07:37:561
1648499107,2cyclictest19739-21sed08:20:281
1648499107,2cyclictest16457-21head08:15:171
1648499100,9cyclictest0-21swapper/107:20:171
1648299108,1cyclictest7131-21seq09:05:010
1648299108,1cyclictest32346-21ntpq07:40:200
1648299108,1cyclictest25250-21apache_volume08:35:130
1648299108,1cyclictest24153-21cut07:25:150
1648299107,2cyclictest22593-21grep07:20:210
1648299107,2cyclictest14765-21cat08:10:260
164889997,1cyclictest13883-21sed08:10:152
164889992,3cyclictest225-21systemd-journal08:00:002
164849990,8cyclictest0-21swapper/108:25:121
164829996,2cyclictest5495-21grep09:00:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional